DOCUMENT RESUME 



ED 475 532 



HE 035 769 



AUTHOR 

TITLE 

INSTITUTION 



ISBN . 

PUB DfCTE 
NOTE ^ 

AVAILABLE FROM 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 

IDENTIFIERS 



Campbell, Carolyn; Rozsnyai, Christina 

Quality Assurance and the Development of Course Programmes. 
Papers on Higher Education. 

United Nations Educational, Scientific, and Cultural 
Organization, Bucharest (Romania) . European Centre for Higher 
Education. 

ISBN-92-9069-171-3 

2002 - 00-00 

218p.; For the four-volume series, see HE 035 767-770. 

Carfax Publishing, Taylor & Francis Ltd., Customer Services 
Department, 325 Chesnut Street, 8th Floor, Philadelphia, PA 
19106 ($15). Tel: 800-354-1420 (Toll Free); Fax: 215-625- 
2940; Web site: http://www.cepes.ro. 

Books (010 ) — Reports - Descriptive (141) 

EDRS Price MF01/PC09 Plus Postage. 

Foreign Countries; ^Higher Education; *Institutional 
Evaluation; Program Evaluation; Quality Control 
*Europe; ^Quality Assurance 



ABSTRACT 

This volume is the fourth in a series related to a project to 
create a Regional University Network on Governance and Management of Higher 
Education in South East Europe. It is both an essay on and source manual for 
quality assurance in higher education. Part 1 describes the origins and 
concepts of quality assurance systems that have been emerging in higher 
education throughout Europe, North America, and elsewhere. Examples are 
chosen from Western, Central, and Eastern Europe. The question of program 
evaluation versus institutional evaluation is explored, as is the question of 
having one or several evaluation and assessment agencies for any one country. 
Part 2 contains five "annexes" on topical issues, mostly from official 
sources, that illustrate how different quality assurance organizations 
operate. Part 3 contains a set of specialized bibliographies on the subject 
of quality assurance in higher education. (SLD) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 




CEPES 



Papers on Higher Education 




permission to reproduce and 

DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 




Quality Assurance and the Development 
of Course Programmes 



Carolyn Campbell and Christina Rozsnyai 






UNESCO - CEPES 



Authors are responsible for the choice and the presentation of the 
facts contained in the Papers on Higher Education and for the 
opinions expressed therein, which are not necessarily those of 
UNESCO and do not commit the Organization. 

The designations employed and the presentation of the material 
do not imply the expression of any opinion whatsoever on the part 
of UNESCO concerning the legal status of any country, territory, 
city, or of its authorities, or concerning the delimitation of its 
frontiers or boundaries. 



Papers on Higher Education 




CEPES 



Regional Univenity Network 
on Governance and Management 
of Higher Education in South Eau Europe 




Quality Assurance and the 
Development of Course Programmes 



Carolyn Campbell and Christina Rozsnyai 





Bucharest 

2002 




Papers on Higher Education 



Editor of the Series: 

Leland Conley Barrows 



Assistants to the Editor: 



Maria-Ana Dumitrescu 
Valentina nslaru 
Viorica Popa 



ISBN 92-9069-171-3 
© UNESCO 2002 



Contents 



Preface 9 

PART I. UNDERSTANDING THE PHENOMENON OF 

QUALITY ASSURANCE 13 



Chapter 1 . Background - The Changing Context of 

Higher Education 15 

1.1. The Changing Environment for Higher Education - 

A Driver for the Emergence and Evolution of National 
Quality Assurance Systems 15 

Chapter 2. Concepts of and Approaches to Quality 19 

2.1. Concepts of Quality 19 

2.2. Approaches to Quality and Standards 22 

2.3. Establishing a Regulatory Framework for Quality: 

A Role for Government 24 

2.4. The Purpose of Quality Assurance 25 

2.5. The Scope of quality assurance 26 

2.6. The Cost of Evaluation 27 

2.7. What Makes a “Good” External Quality Assurance 

system? 28 

Chaptet 3. Key Terms 31 



Chapter 4. Quality Assurance and Programme Design 

in Europe 35 

4.1. Introduction 35 

4.2. Quality Assurance and Programme Design in 
Western Europe: The Evolution and Current State 

of Affairs - Description and Comparative Analysis 36 

4.2.1. Increased Institutional Autonomy in Return 

for Greater Accountability 37 



4.2.2. Diversification and Expansion 37 

4.2.3. Need for New Methods of Teaching, Learning, 

and Assessment 38 

4.2.4. Internationalization ....38 

4.2.5. Concepts of and Approaches to Quality 38 

4.2.6. Purposes of E^^aluation 40 

4.2.7. Scope of Evaluation: Institutional or 

Programmatic - Universities or Colleges 42 

4.2.8. Institutional Evaluation 44 

4.2.9. Thematic Reviews and Reports 46 

4.2. 10. One Agency or Several 46 

4.2. 1 1 . Evaluation Criteria and Standards 47 

4.2. 12. Evaluation Methods and Instruments 48 

4.2.13. The Self-Evaluation Study 48 

4.2. 14. External Experts: Selection, Training, and 

Briefing 50 

4.2. 15. Reports and Judgments 51 

4.2. 16. Aspects of Programme Design 52 

4.2. 17. Two Cycles of Education 53 

4.2. 18. Emerging Trends in Quality Assurance 
and the Design of Programmes in Western 

Europe 54 



4.3. Quality Assurance and Programme Design in 
the Central and Eastern European Countries: 
The Evolution and Current State of Affairs - 



Description and Comparative Analysis 59 

4.3.1. Methodology 66 

4.3.2. Programme Design 67 

4.3.3. Levels of Study 68 

4.3.4. The Legal Setting for Programme Design 74 

4.3.5. Quality Assurance 77 

4.3.6. Quality Assurance Co-operation between 

Central and Eastern European Countries 80 

4.3.7. Expected Trends 82 



o 



7 



PARTII. ANNEXES 



85 



Annex 1 Resources Chart from the December 2001 

INQAAHE Survey 87 

Annex 2 Descriptions of Higher Education Institutions and 
Quality Assurance in Central and Eastern European 
Countries 89 

2.1. Albania 89 

2.2. Bulgaria 90 

2.3. Croatia 92 

2.4. The Czech Republic 93 

2.5. Estonia 95 

2.6. Hungary 97 

2.7. Latvia 99 

2.8. Lithuania 101 

2.9. Macedonia 103 

2.10. Poland 105 

2.11. Romania 108 

2. 12. The Russian Federation 109 

2.13. Slovakia 112 

2.14. Slovenia 114 

2.15. Ukraine 116 

2. 16. Country Overview of Accreditation and Quality 

Assessment Agencies 118 

2. 16. 1 . Clarification of Terminology 1 18 

2.16.2. Schematic Presentation of Accreditation 
and Quality Assessment Agencies by Central 

and Eastern European country 119 

2. 16.3. Internet Links for Quality Assurance in the 

Central and Eastern European Countries 127 

2. 16.4. Questiormaire sent to Quality Assurance 

Agencies in Fifteen Central and Eastern 
European Countries 128 



Annex 3. Glossary 131 

Annex 4. Quality Assurance: Aspects of Processes, 

Approaches, and Networking 137 

4.1. Quality Assurance Agencies: Principles, Values, 

and Process Standards 138 

4.1.1. FH Conference/FH Council Austria: External 

Evaluation of Fachhochschulen in Austria 138 

4.1.2. Denmark: Danmarks E^^alueringsinsitut 

(EVA) 139 

4.1.3. Germany: Accreditation, Certification, and 

Quality Assurance Institute (ACQUIN) 140 

4.1.4. The United Kingdom: The Quality Assurance 

Agency for Higher Education (QAA) 140 

4.1.5. United States of America: CHEA 142 

4.2. Quality Assurance: Self-Evaluation Documents 

and Reports 143 

4.2.1. United Kingdom: Guidance to Institutions on 

Producing a Self-E^^aluation - An Example from 
the Draft Handbook on Institutional Audit in 
England 143 

4.2.2. Germany: FH Conference/FH Council: 

Quality System 146 

4.3. The Selection, Appointment, and Training of Peers: 

Some Examples of Current Practice 150 

4.3.1. Austria: FH Conference/FH Council 150 

4.3.2. Denmark: Evaluation Group (EVA) 152 

4.3.3. The United Kingdom: The Quality Assurance ...152 

4.4. International Benchmarking and Evaluation: 

The Danish Evaluation Agency (EVA) - An 
International Comparative Evaluation of 
Programmes in Agricultural Science 153 

4.5. Approach to Quality Assurance: The Example 

of an Agency in a Central and Eastern European 
Country 157 

4.6. Quality Assurance from the Perspective of 
a University Network Association: A Policy 
Statement of the European University Association 
(EUA) 



161 



4.7. Accreditation and Recognition: Some International 

Initiatives by the Engineering Profession 167 

4.7.1. The European Federation of National 

Engineering Associations (FEANI) 167 

4.7.2. The Washington Accord 168 

4.8. What Do Quality Assurance Networks Do? 

The European Network for Quality Assurance 

in Higher Education (ENQHA) 169 

Annex 5. The Design of Study Programmes: 

Establishing External Reference Points for Quality 

and Standards 173 

5. 1. The Development of External Reference Points 

for Quality and Standards of Study Programmes 174 

5.1.1. Towards Shared Descriptors for Bachelor’s 
and Master’s Degrees: An International Approach 
- A Report from the Joint Quality Initiative 

Group 174 

5.2. The Design of Study Programmes 178 

5.2.1. The Central Evaluation and Accreditation 

Agency, Hannover, Germany (ZevA) 179 

5.3. Towards Subject Specific Descriptors for 
Qualifications: A National Approach - The 
Quality Assurance Agency for Higher Education 

of the United Kingdom 187 

5.4. Tuning Structures in Higher Education: Lessons 

Learned in how to Achieve Results 189 



PART III. REFERENCES, BIBLIOGRAPHIES, AND 

WEB RESOURCES 193 

1 . List of References 1 93 

2. Bibliographies 197 

2.1. Central and Eastern Europe: Useful Publications on a 

Country by Country Basis 197 

2.2. General Readings or Readings Pertinent to all the 

Central and Eastern European Countries 205 

2.3. Thematic Bibliographies 207 

2.3.1. The Changing Context of Higher Education: 

Trends, Challenges, and Responses 207 

2.3.2 Governance and Management in Higher 

Education 209 

2.3.3. Income Generation and Entrepreneurship 210 

2.3.4. The Evaluation of Institutional Performance....21 1 

2.3.5. Human Resource Management 212 

2.3.6. Quality Assurance and Management 212 

2.3.7. Open and Distance Learning 216 

3. Some Useful Websites 217 

3.1. Intergovernmental Organizations 217 

3.2. Non-Govemmental Organizations 218 

3.3. Subject Accreditation/Recognition Agencies 

Operating on an International Basis 218 

3.3.1. Business and Management 218 

3.3.2. Engineering.. 219 

3.4. Some Other Resources on Aspects of Quality 

and/or Education 219 

The Authors .221 



UNESCO-CEPES Publications 



227 



Preface 



This publication is the fourth realization in a series of four 
publications on aspects of governance in higher education that 
are being produced as a partial outcome of the joint UNESCO- 
CEPES - European Commission project to create a Regional 
University Network on Governance and Management of Higher 
Education in South East Europe. The Programme was 
originally presented through Table One “Democracy and Good 
Governance” of the Stability Pact for South East Europe as 
part of its “quick-start package”. It has been developed through 
the Task Force on Education and Youth, Enhanced Graz 
Process, a coordinating mechanism for educational co- 
operation with South East Europe. 

The basic assumption of the Programme is that, when 
considering the overall situation in the countries of the region, 
education in general - higher education in particular - should 
play a key role in supporting the search for sustainable peace, 
reconciliation, and development of civil society. 

Its wider objectives include the following: 

- integration of the universities and higher education 
authorities of Southeastern Europe into existing 
European networks; 

- development of higher education policies that are based 
on European standards and international best practice 
in the areas of strategic management, financial 
management, relations with civil society, and quality 
assurance; 

- development of national and institutional capacities and 
skills in higher education strategic management and 
policy making; 

- stimulation of the establishment and/or consolidation of 
new structures and mechanisms of financial 
management, based on the principles of university 
autonomy and accountability, while encouraging the 
establishment of links with civil society and local 
economies. 
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The anticipated outcomes of the programme are expected to 
include the following: (i) integration of the countries of South 
Eastern Europe into the European Higher Education Area as 
defined in the Bologna Declaration; (ii) the creation of a 
network of the authorities and institutions involved in higher 
education through which good practice in academic governance, 
policy making, strategic and financial management, and 
quality assurance in higher education can be exchanged; (ui) 
strengthened national institutional capacities and skills in 
regard to strategic management and policy making in higher 
education; (iv) the creation of new structures and mechanisms 
for financial management, based on the principles of university 
autonomy and accountability, while encouraging links with 
civil society and local economies. 

This volume, that brings the series to a close, is both an 
extended essay and a source manual on the necessity for good 
quality assurance mechanisms in higher education. Its 
authors, Carol 5 m Campbell of the University of Surrey in the 
United Kingdom, and Christina Rozsnyai of the Hungarian 
Accreditation Committee, are experts in the field. 

Part I of the volume describes the origins and the concepts 
of the quality assurance systems that have been emerging in 
higher education throughout Europe, North America, and 
elsewhere. Examples are chosen from Western Europe and 
from Central and Eastern Europe. The question of programme 
assessment and evaluation versus institutional evaluation is 
discussed as is the question of one or several evaluation and 
assessment agencies for any one country. 

Part II of the volume consists of a number of readings on 
topical issues drawn from a variety of mostly official sources 
illustrating how different quality assurance organizations in 
different countries operate. In addition, a glossary of frequently 
used terms is included as well as a set of short descriptions of 
the quality assurance institutions in fifteen Central and 
Eastern European countries. 

In addition to listing the references cited in Parts I and II of 
the volume. Part III includes a set of specialized bibliographies 
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on the subject of quality assurance in higher education. Some 
of the publications in one or more of the bibliographies are 
followed by short summaries. The Websites of the major 
quality assurance agencies- in Europe and America are also 
listed. 

In short, this publication will give the reader a very 
comprehensive overview of the phenomenon of quality 
assurance in higher education. It, along with the other 
volumes: No. 1, a survey of governance structures of higher 
education in several countries of South East Europe (with 
suggestions for improvement). No. 2, an extended essay on 
strategic planning and management of higher education, and 
No. 3, a study of financial management in higher education, 
particularly formula funding, provide a comprehensive set of 
suggestions as to how best to approach the global reform of 
governance in higher education. 

The four volumes have been published in the UNESCO- 
CEPES series. Papers on Higher Education. Our hope is that 
they will contribute strongly to the anticipated goal of creating 
a successful Regional University Network of Governance and 
Management of Higher Education in South East Europe. 



Jan Sadlak 
Director of UNESCO-CEPES 



Parti 



Understanding the Phenomenon 
of Quality Assurance 



Chapter 1 



Background - The Changing Context 
of Higher Education 



Quality assurance in higher education was until relatively 
recently an implicit activity. Arguments that quality could not 
be measured but could be recognized by academics when and 
where it existed were prevalent. However, over the last two 
decades, a number of factors have combined to challenge 
traditional views about quality in higher education and how it 
is assured. These factors have been elaborated by many 
commentators - individuals and organizations such as 
UNESCO and the World Bank - and have led to the making of 
quality assurance in higher education “a central objective of 
governmental policies and an important steering mechanism in 
higher education systems worldwide” (Van Damme, 2002, p. 6). 

In the rapidly changing environment of higher education, 
the maintenance of high quality and standards in education 
has become a major concern for higher education institutions 
and governments; thus, the demand for explicit quality 
evaluation and assurance processes has increased. The result 
has been the introduction of national quality assurance 
systems into many countries and the planned introduction of 
such systems into other countries. 

The challenges facing higher education worldwide include 
the following: 

- the need to assure quality and standards against a 
background of substantially increased participation - a 
process often referred to as the massiflcation of higher 
education. This process accelerated throughout the latter 
part of the Twentieth Century as many countries began 
to consider that their economic and social future was 
dependent, in part, on the availability of quality higher 
education for the majority of the population rather than 
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for a small elite. However, expansion has not always 
been well planned or controlled; 

- the expansion in student numbers with either constant 
or declining (public) funding resulting in a lower unit of 
resource per student. This position has been 
compounded by the inefficient use of available resources. 
Examples of inefficiencies include overly high staff- 
student ratios, programme duplication in many small 
tnstitutions/units with high unit costs, and under- 
utilized facilities. Such inefficiencies divert resources 
from such nbjectives as quality and access (World Bank, 
2002); 

- increased demand for accountability in higher education 
institutions as a result of deregulation and the granting 
of increased autonomy in regard to such matters as 
curriculum design, the selection of students, and the 
appointment of staff. However, increased autonomy has 
not always been accompanied either by financial 
authority or by improved institutional management and 
strategic planning capabilities; 

- the meeting of new expectations in terms of the 
“employability” of graduates in the knowledge society; 

- the addressing of demands from a variety of stakeholders 
for increased and improved information about 
programmes and institutions and about the skills, 
competencies, and aptitudes which graduates possess; 

- the contribution to the achievement of social and 
political agendas such as access, inclusion, and equity. 

In addition to these factors, recent developments include 
the appearance of new providers of tertiary education, 
sometimes in competition with traditional public higher 
education, and new modes of provision, such as on-line 
learning, resulting from the information and communication 
technology revolution. An example of this type of competition is 
reflected in the “new technologies” and the rise of the 
“Academies” of Microsoft, Cisco, SAP, etc., that have created a 
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parallel universe of IT qualifications and standards with global 
coverage (Adelman, 2000). 

Quality in higher education is not only a national concern 
but has become an international issue through academic, 
political, and commercial developments associated with 
globalization, such as the rise of market forces in tertiary 
education and the emergence of a global market for skilled 
professionals and graduates. In some countries, the traditional 
providers of higher education are facing competition from 
transnational education providers as well as from the 
emergence of local commercial providers. Through the 
internationalization of higher education national systems, 
qualifications and individual higher education institutions 
have become exposed to the wider world. This exposure has 
stimulated a demand for better information and transparency 
about quality and standards in order to attract and retain 
students and staff, both national and international students, 
and to secure the recognition of qualifications. 

Quality assurance is a central thrust in the process of 
change in European higher education following the signing of 
the Bologna Declaration and the Prague Communique 
(<http://www.msmt.cz/summit/prague>), and has been 
highlighted as a policy implication in the discussions being 
sponsored by the Global Agreement on Trade in Services 
(GATS) on the further liberalization of the trade in education 
services (<http://www.wto.org>). 



Chapter 2 

Concepts of and Approaches to Quality 



2.1. CONCEPTS OF QUALITY 



There are many different understandings of the term, quality, 
often reflecting the interests of different constituencies or 
stakeholders in higfier education. Thus, quality is a 
multidimensional and often a subjective concept (PHARE 
Multicountiy Handbook, 1998). 

Conceptions of quality were categorized by Harvey and 
Green (1993) and were elaborated in the PHARE Manual of 
Quality Assurance: Procedures and Practices (1998). They 
include the following: 

- Quality as excellence. This definition is considered to be 
the traditional academic view that holds as its goal to be 
the best. 

- Quality as ''zero errors”. The idea of “zero errors” is 
defined most easily in mass industry in which product 
specifications can be established in detail, and 
standardized measurements of uniform products can 
show conformity to them. As the “products” of higher 
education, the graduates, are not expected to be 
identical, this view is not always considered to be 
applicable to higfier education. 

- Quality as ''fitness for purpose”. This view requires that 
the product or service meet a customer’s needs, 
requirements, or desires. Learners (students) and 
prospective learners, those who fund higfier education, 
the academic community, government, and society at 
large are to a greater or lesser extent all clients or users 
of higher education but may have very different views of 
both “purpose” and “fitness”. 

A major weakness of the “fitness for purpose” concept 
is that it may seem to imply that “anything goes” in 
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higher education so long as a purpose can be formulated 
for it. This weakness is more likely to be exacerbated in 
large and diverse higher education systems in which a 
wide range of “purposes” at institutional level may be 
identified by individual institutions, generally through 
their mission statements, and at more precise academic 
levels through the learning outcomes of particular 
programmes. This diversity is often further complicated in 
transnational and distance education (situations in which 
educational provision crosses borders) as there is 
frequently a divergence of national views between 
“sending” and “receiving” countries as to both “fitness” 
and “purpose”. 

By complementing “fitness for purpose” with a notion of 
“fitness of purpose”, an evaluation can consider and 
challenge the comprehensiveness and relevance of purposes 
in order to ensure Improvements. 

- Quolity as transforniation. This concept focuses firmly on 
the learners: the better the higher education institution, 
the more it achieves the goal of empowering students 
with specific skills, knowledge, and attitudes which 
enable them to live and work in the knowledge society. 
This notion of quality may be particularly appropriate 
when there have been significant changes in the profile 
of learners, for example, when changes in society or 
politics have enhanced access to higher education for 
large numbers of disadvantaged learners. It is argued 
that the delivery of a transformational quality approach 
involves five key elements (Harvey and Knight, 1996, p. 
117): 

• envisioning quality as a transformational process 
designed to enhance the experience of students; 

• a bottom-up approach to continuous improvement; 

• responsiveness and openness as the means of gaining 
greater trust; 

• an emphasis on effective action; 
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• external monitoring which is sensitive to internal 

procedures (and values). 

While this notion is popular, it may be- difficult to 
measure quality as transformation in terms of 
intellectual capital (Lomas, 2002). 

Quality as threshold. Defining a threshold for quality 
means setting certain norms and criteria. Any 
programme, department, or Institution, which reaches 
these norms and criteria, is deemed to be of quality. The 
advantage of setting a threshold is that it is objective and 
certifiable. However, there are arguments that setting a 
threshold creates uniformity across the higher education 
system. This argument might well apply if institutions 
adopt a “compliance” mentality and only do what is 
sufficient to satisfy the minimum. There are significant 
disadvantages to this concept, especially when the 
criteria and standards are based on quantitative “input” 
factors enshrined in law. It camnot readily be adapted to 
changing circumstances or to stimulate change and 
innovation. In this respect, the “threshold” can mitigate 
against improvement. Neither does it take account of 
“output” standards, the actual level of achievement by 
graduates, the criteria used to assess these 
achievements, and how that assessment is verified. 
Nevertheless, in many European higher education 
systems, a “minimum standards” variant has been used 
if only as a starting point in the quest for quality. 

Quality as value for money. The notion of accountability 
is central to this definition of quality with accountability 
being based on the need for restraint In public 
expenditure (Lomas, 2000). 

Quality as enhancement or improvement This concept 
emphasizes the pursuit of continuous improvement and 
is predicated on the notion that achieving quality is 
central to the academic ethos and that it is academics 
themselves who know best what quality is at any point 
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in time. Disadvantages of this concept are that it is 
difficult to “measure” improvement and that the evidence 
of improvement may not be easily discernible to the 
outside world. 

Some of these concepts of quality still hold true especially 
when explicit quality assurance and accreditation procedures 
are being developed and introduced for the first time either at 
system or at institutional level. But, notions of quality are 
evolving or merging, either as the result of the changing 
context in which higher education institutions are operating in 
some countries, or as a result of growing expertise within 
higher education systems and institutions in devising their 
own concepts of quality and models of evaluation and quality 
management. Mismatches between the requirements of the 
external quality assurance agency and institutional 
approaches to quality can be a cause of tension in relations. 

2.2. APPROACHES TO QUALITY AND STANDARDS 

Some interest has been shown in the application of quality 
processes and standards adapted from other sectors, for 
example, ISO 9001:2000 standards, to higher education 
institutions. The International Organization for Standardization 
(ISO) <http://www.iso.ch/en> is a worldwide federation of 
national standards bodies from 140 countries. Established in 
1947, it is a non-govemmental organization with the mission 
to promote the development of standardization and related 
activities in the world with a view to facilitating the 
international exchange of goods and services and to developing 
co-operation in the spheres of intellectual, scientific, 
technological, and economic activity. ISO has developed 
standards and a process for certifying that organizations have 
achieved them. 

ISO has been described as requiring organizations “to say 
what they do, do what they say, then prove it to a third party” 
(Seddon, 2000). Universities have tended to seek this 
certification for aspects of their non-academic services such as 
conference facilities, such student services as accommodation 
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and careers advice and individual academic departments, 
rather than for their internal quality monitoring or academic 
processes as a whole. One European Quality Assurance 
Agency [Agenda per la Qualitat del Sistema Universitaria 
Cataluyna (<http://www.agenqua.org>) has sought and gained 
ISO 9002 Certification for its own processes. A criticism of the 
ISO approach has been of its perceived focus on process rather 
than on outcomes. This criticism is apparently being 
addressed in the new process standards 9001:2000 which 
feature an increased focus on customer satisfaction. 

The continuous quality improvement model of the 
European Foundation for Quality Management (EFQM), the 
“Excellence” model, is another example of a standard for 
business and industry which has developed a variant to be 
applied to public sector services (<http://www.efqm.org/ 
new_website/>). The model is being used by several 
universities in the United Kingdom, (<http://www. excellence. 
shu.ac.uk>), Germany, and Turkey, and the second round of 
institutional accreditation in Hungary will be based on this 
model adapted to local needs. The impetus to establish EFQM 
came from a desire to develop a European framework for 
quality improvement, based on Total Quality Management 
(TQM) principles, along the lines of the Malcolm Baldridge 
Model in the USA and the Deming Prize in Japan, (see 
<http : / /WWW. qpronline . com/baldridge /index. html> for further 
information). 

The EFQM also supports benchmarking exercises that it 
views as a powerful tool in the quest for continuous 
improvement. The European Benchmarking Code of Conduct, 
a guide intended to advance the professionalism and 
effectiveness of benchmarking in Europe, is published by 
EFQM. Higher education institutions in some countries are 
increasingly interested in benchmarking. This interest is 
reflected in the purposes and aims of some of the national 
evaluation agencies (see below) and has been the topic of 
recent workshops organized by the European Network for 
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Quality Assurance (ENQA - see <http://www.enqan.net> for 
details). 

Developments in new forms of provision in higher education 
such as on-line or “e” -learning and cross border or 
transnational education are prompting the articulation of new 
standards and criteria at both national and institutional level 
in higher education. Conflicting views exist as to whether or 
not the same standards and evaluation approaches can be 
used for different segments of the tertiaiy education system 
and whether they can be applied to traditional and new forms 
of higher education provision alike. For those for whom the 
answer is yes, the way forward is viewed as a focus on deflning 
academic standards and qualiflcations in terms of learning 
outcomes and competencies rather than as a focus on 
traditional input and process factors. Arguments for embracing 
the same standards and qualiflcations include the desire to 
accommodate diversity and to facilitate lifelong learning 
without creating a confusing jungle of separate qualifications. 
Whatever view is taken, a consensus is emerging about the 
need for clearer standards and criteria as well as for better 
information and disclosure about institutional and student 
performance. 

Whatever concept of quality is adopted by a national 
system, the evaluation procedures introduced by the external 
evaluation agency must match it. The same principle applies at 
institutional level as well, but all institutions within a system 
need not adopt the same approach to quality in any one 
system. 

It has recently been observed that, “twenty years of 
operational expertise in quality assurance in higher education 
has not led to a consensus on how the concept of quality 
should be defined, rather the opposite” (Van Damme, 2002). 

2.3. ESTABLISHING A REGULATORY FRAMEWORK FOR 
QUALITY: A ROLE FOR GOVERNMENT 

The external evaluation of the quality of higher education 
provision began more than a hundred years ago, with the 
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introduction of accreditation in the United States of America. 
However, accreditation in the United States was and still is 
organized by non-govemmental organizations which either 
accredit institutions (the “regional” accreditors and some 
national accreditors) or programmes (the specialized accreditors 
and some national accreditors). While accreditation is voluntary 
and non-govemmental, it is, nevertheless, a requirement for 
access to Federal government funding such as student loans 
(through recognition of the accreditors by the United States 
Department of Ekiucation), and may be a requirement for the 
licensing of professionals by authorities in individual States. 
Accreditation has facilitated - but not necessarily guaranteed - 
credit transfer between institutions and also student admission 
to graduate programmes. The Council for Higher Education 
Accreditation (CHEA <http://www.chea.org>) is a non- 
govemmental body which carries out the recognition of 
accreditation agencies in the United States and fosters research 
on aspects of accreditation and related quality assurance 
matters. 

The American approach to quality in higher education 
contrasted with early approaches to quality in Europe which, 
in the main, were based on legislation and government control 
over higher education institutions and the design of 
programmes. However, many European governments have 
determined that traditional academic controls and legislation 
are not effective in dealing with the range of challenges faced 
by higher education and have either introduced or facilitated 
the establishment of explicit quality assurance systems. More 
detailed commentary on and analysis of the development of 
external quality assurance in Western and Central and 
Eastern Europe follows below. 

2.4. THE PURPOSE OF QUALITY ASSURANCE 

At system level, the purpose of quality assurance has to be 
determined and communicated to higher education 
institutions. There is continuous debate as to whether the 
emphasis of external quality assurance should be on 
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accountability or on improvement and how an appropriate 
balance between these two purposes might be struck. It might 
be appropriate in certain circumstances, for instance, if 
addressing the rapid growth of unregulated private education 
or the introduction of new types of institutions or 
qualifications, to put an emphasis on accountability and 
compliance. However, as institutions develop more effective 
and sophisticated internal quality assurance mechanisms, 
pressure will grow to move the balance from compliance to 
improvement. Moreover, if external requirements, for example, 
in respect of programme approval and design are very rigid, 
there may be limited scope to demonstrate improvement and 
institutions may not have the flexibility to respond either 
rapidly or in an innovative manner to new demands. 

2.5. THE SCOPE OF QUALITY ASSURANCE 

The scope of quality assurance is often determined by the 
shape and size of the higher education system itself. The 
tendency is for a single national agency to evaluate all higher 
education provision (both university and non-university 
provision), but there are some notable exceptions of countries 
in which there are separate agencies that are responsible for 
different types of programmes, institutions, regions, or 
purposes. These exceptions reflect not only the different stages 
of development within higher education systems but also 
political and cultural preferences (World Bank, 2002). 
Definitions of the scope of quality assurance can be narrow or 
broad. An example of the latter, that is, a broad scope^ might 
include the following dimensions (Middlehurst, 2001; ENQA): 

- regulation: including legal frameworks, governance, 
responsibilities, and accountabilities; 

- educational process: admission/ selection of students, 
registration or enrollment, curriculum design and 
delivery, support for learning, assessment of students; 

- curriculum design and content approval and/or 
accreditation levels and standards; 
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- learning experience: user protection, student experience, 
complaints, and appeals; 

- outcomes: qualifications, certificates, transcripts, the 
Diploma Supplement, transferability, recognition, and 
value. 

The respective roles of the national system and institutions 
need to be addressed. While both may have a role in all of 
these dimensions of quality assurance, it may not be of equal 
weight. 

2.6. THE COST OF EVALUATION 

Cost may be an important consideration in determining factors 
such as the focus, scope, and purposes of an external 
evaluation system. Costs include not only those of the setting 
up of the agency and of the operation of the external process 
but also the hidden costs to institutions. The latter include 
staff time in preparing for external evaluation and the 
collection and collation of information for the self-evaluation 
and have to be taken into account in determining the type and 
amount of information to be provided by institutions to 
external agencies. The more such information can be drawn 
from that which already exists within institutions or which 
would be of use to institutions for their own internal quality 
systems, the better. Other factors, which might be taken into 
account in deciding on the focus and scope of evaluation, 
include: 

- the number and types of institutions in the national 
system; 

- whether or not institutions, individual programmes, or 
broader groupings of subject/ disciplines are to be the 
focus. The costs of either evaluating or accrediting on a 
programme-by-programme basis may be substantial. It 
is worth noting that countries such as the Netherlands 
and Germany are, or will be, using this basis for the 
accreditation of their new Bachelor’s and Master’s 
Degrees. This situation contrasts with that prevailing in 
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the United States, in which accreditation of individual 
programmes only applies in areas such as business and 
management, health and medicine, technology and 
engineering, or for certain types of provision, e.g., 
distance learning; 

- the frequency of evaluations - a national cycle at 
programme level or on an od hoc/ on demand basis? The 
length of cycles of evaluation tends to vary but is never 
less than three years (unless a serious problem has been 
identified) and can extend up to ten years; 

- the remuneration of experts. In many systems, experts 
are recruited on a “volunteer” basis and receive only 
reimbursement for expenses for travel and subsistence; 
however, in others, there may be some kind of 
honorarium or indeed some compensation paid to the 
expert’s own institution for the time spent in activities as 
an external expert for the external agency. 

These factors all have cost implications, but an advantage 
of smaller systems is that universal visiting at 
programme /discipline level is potentially more affordable than 
in large higher education systems. 

The means by which the activities of evaluation agencies in 
Europe are funded vary from receipt of a direct subvention 
from the Ministry of Education to a mixture of subscription 
incomes from higher education institutions and from contracts 
or evaluation or accreditation fees. However, a common feature 
is that agencies have autonomy over the use of their funds. 

2.7. WHAT MAKES A “GOOD” EXTERNAL QUALITY 
ASSURANCE SYSTEM? 

A consensus is emerging as to what constitutes an appropriate 
system of external evaluation, the core process elements of 
which include: 

- reliance on autonomous or semi-autonomous agencies. 
This reliance gives legitimacy to internal review processes 
(EUA policy statement 2001, in Annex 4.6); 
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- agreement on explicit standards and expectations; 

- a self-evaluation study by the academic department, 
faculty, or institution; 

- an external review conducted by visiting experts; 

- a written report or recommendations; 

- public reporting of the results; 

- recognition that the evaluation process in itself is at least 
as important as the results (El Khawas, 1998). 

Other success factors include consultation and dialogue 
with stakeholders and higher education institutions by the 
external agency when developing processes and criteria, that 
is, before setting a system in action. This type of dialogue is 
considered essential to achieving meaningful outcomes to the 
evaluation process (Franke, 2002). Programme accreditation/ 
evaluation benefits from clarity and a focus on teaching and 
learning processes, practices, and performance (Genis, 2002). 

A good system for quality assurance should meet its desired 
objectives without unnecessary bureaucracy or intrusion in 
the primary activities of an institution (Middlehurst, 2001). 
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Key Terms 



The vocabulary of “quality” is often jargon-ridden and arcane 
with the same words being used to describe different 
processes. A glossary of terms is included in Annex 3; however, 
an abridged version, expanding on certain key terms, follows: 
Accreditation is defined in many ways. Three examples are: 

- “Accreditation is a formal, published statement regarding 
the quality of an institution or a programme, following a 
cyclical evaluation based on agreed standards” (CRE, 
now EUA, 2001). 

- “Accreditation is a process of external quality review 
used by higher education to scrutinize colleges, 
universities, and higher education programmes for 
quality assurance and quality improvement” (CHEA, 
2000 ). 

- “Accreditation is the award of a status. Accreditation as a 
process is generally based on the application of 
predefined standards. It is primarily an outcome of 
evaluation” (The European Training Foundation, 1998). 

While accreditation has different definitions, forms, and 
functions, it generally has the following as characteristics: 

- It provides [proof] (or not) that a certain standard is 
being met in a higher education course, programme, or 
institution. The standard met can either be a minimum 
standard or a standard of excellence. 

- It involves a benchmarking assessment. 

- Judgments are based solely on quality criteria, never on 
political characteristics, and are always yes /no. 

- The emphasis is on accountability. 
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A benchmark is a reference point or criterion by which to 
measure something. 

Benchmarking is a process of identifying and learning from 
good practices in other organizations. 

(Subject) benchmark statements represent general expectations 
about standards (levels of student attainment) at a given 
level in a particular subject area. They are reference 
points in a quality assurance framework rather than 
prescriptive statements about curricula. (Examples of 
nation^ subject benchmark statements are available at 
<http://www.qaa.ac.uk>, and international developments 
are available at <http://www.relint.deusto.es/Tuning>; 
also, see Annex 5.4. 

A binary system is a dual system of higher education with a 
traditional university sector and a separate and distinct 
non-university sector. 

Evaluation is a general term denoting any process leading to 
judgments and/or recommendations regarding the or 
qu^ty of a unit. (A unit is an institution, programme, 
discipline.) Eyaluation can be an internal process - self- 
evaluation - or an external one conducted by external 
experts, peers, or inspectors. 

An institutional audit is concerned with the processes whereby 
an institution assures quality assurance and quality 
enhancement. An underl 3 dng theme in quality audits is 
the question: how does an institution know that the 
standards and objectives it has set for itself are being 
met? On what evidence is the assessment of the quality 
of its work based, and are there procedures in place to 
ensure that the significant processes are followed up and 
continuously improved? 

Institutional review addresses the ultimate responsibility for 
the management of quality and standards that rests with 
an institution as a whole. 

Licensing is the awarding of permission to operate either a new 
higher education institution or a new course programme 
based on ex ante evaluation of plans. 
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Quality assessment, quality measurement, and review of 
quality are all taken here to be S 5 monymous with 
evaluation, especially when there is an external element 
to the procedure. 

Quality assurance is an all-embracing term covering all the 
policies, processes, and actions through which the 
quality of higher education is maintained and developed. 




Chapter 4 



Quality Assurance and Programme Design 

in Europe 



4.1. INTRODUCTION 

The changing context of higher education worldwide has been 
outlined above in general terms. Over the last century, many of 
the drivers, which prompted the development of external 
evaluation of higher education in Western Europe, had their 
counterparts in the Central and Eastern countries, even if the 
motives were, for the most part, dissimilar. The following 
sections describe aspects of existing quality assurance 
mechanisms in higher education in Europe. They deal, 
respectively, with Western Europe, mainly describing 
developments in the European Union, and with Central and 
Eastern Europe. 

This division was not made to create a new barrier between 
East and West, but rather to allow space to describe and 
reflect on the different starting points, pace, and modes by 
which higher education systems and institutions are travelling 
towards the same destination - the European Higher 
Education Area. Although Sections 4.2 and 4.3 cover similar 
topics, the emphases are different. Section 4.3 on 
developments in Central and Eastern Europe is complemented 
by a detailed annex (Annex 2) describing the higher education 
systems of the region and a first, comprehensive, updating of 
information on the quality assurance agencies in Central 
Europe, based on a questionnaire (Annex 2.16.4) sent to 
members of the new regional network for quality assurance. 
Section 4.2 on Western Europe focuses more on developmental 
work being done at national and international level to establish 
external reference points for quality including generic and 
subject specific descriptors for qualifications. 
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Many of the details in Section 4.2. concerning quality 
assessment and accreditation in the European Union were 
taken from published material, including questionnaires 
completed by members of the International Network for Quality 
Assurance Agencies for Higher Education (<http://www.inqahe.nl>), 
descriptions of their work, in English, provided by agencies on 
their Websites and in publications, from contacts at agencies 
and other academic associations, and from recent seminars 
and workshops on quality and qualifications that have been 
held as part of the Bologna process. A separate questionnaire 
was not sent out to ENQA members as a survey mapping the 
similarities and differences in European quality assurance as 
practiced in the various countries is nearing completion. It is 
being managed by the Danish Evaluation Agency on behalf of 
the European Network for Quality Assurance (ENQA) and 
comprises two phases: 

- A survey of quality assurance in thirty- two countries, 
which will focus on the evaluation methods used in 
them, and country specific reports will be prepared. 

- The second phase of the project is an analytical report on 
the state-of-the-art of evaluation procedures, which will 
be based on a number of identified themes and aspects 
of the evaluation methodologies and procedures 
identified tu the first phase. 

Both reports will be made available on the ENQA web-site 
<http://www.enqa.net>. 

4.2. QUALITY ASSURANCE AND PROGRAMME DESIGN IN 
WESTERN EUROPE: THE EVOLUTION AND CURRENT STATE 
OF AFFAIRS - DESCRIPTION AND COMPARATIVE ANALYSIS 

The drivers, that prompted the introduction of external quality 
assurance systems in the European Union, mirror many of the 
points made above concerning the changing context in which 
higher education was operating: 
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4.2.1. Increased Institutional Autonomy in Return for Greater 
Accountability 

The grant of increased autonomy to higher education was 
paralleled by a rise in the demand for accountability to re- 
assure authorities that resources were being well-spent, a 
value-for-money argument in certain Western countries, and 
to ensure that the expectations of interested constituencies - 
students, would-be-students, their parents, employers, and 
society at large would be met. Increased institutional 
autonomy, however, pointed up the need for enhancement and 
improvements in institutional leadership, information 
gathering and dissemination, and effective decision-making 
within institutions. 

4.2.2. Diversfication and Expansion 

In addition to increased numbers of students resulting in 
increasingly large classes, the diversification of the student 
intake - including more adult learners - raised issues of 
teaching and learning methods and put strains on access to 
learning resources and support. In many countries, greater 
levels of participation in higher education were not always 
matched by a corresponding rate of increase in the number of 
graduates. Significant percentages of European students did 
not complete their studies raising issues such as the waste of 
human talent and potential and of public money. Reasons for 
high drop-out rates included the growing mismatch between 
the official length of studies and the actual time spent gaining 
qualifications - perhaps reflecting problems of overloaded 
curricula, out-dated teaching and learning methods, and the 
lack of intermediate or shorter qualifications in the higher 
education systems in question. The retention and progression 
of students remains a concern in many countries and has 
emerged as a performance indicator in several systems either 
for quality assurance or for funding purposes, e.g., in 
Denmark, in Finland, in the United Kingdom, and in some 
Lander in Germany. 
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Diversity has meant not only the development of new types 
of higher education institutions such as polytechnics [e.g., in 
Portugal and Finland) and other tertiaiy level institutions, but 
also the introduction of new disciplines and programmes, 
including shorter degree programmes in certain systems. The 
wider variety of types of higher education institutions and of 
course programmes has presented challenges for students, 
who have needed more and better information to help them 
make informed choices about the best study options for them, 
and to external evaluation agencies, the criteria of which were 
previously been predicated on traditional university provision. 
Diversity has also highlighted the number of barriers that 
remain in terms of mobility and progression as links and 
pathways between different types of higher education are not 
always defined. 

4.2.3. Need for New Methods of Teaching, Learning, and 
Assessment 

The development and widespread use of ICT applications for 
education has created the need for new skills for teachers and 
has implications for curriculum design and the definition of 
new criteria for the quality assurance of delivery. 

4.2.4. Internationalization 

The mobility of students either in full programmes or in 
exchange schemes has increased demands for the recognition 
of qualifications, credits, and study periods abroad. This 
demand has prompted the need for greater transparency of 
curricula and assessment. 

Not all of these factors have applied at one and the same 
time to all European countries, but they all have a common 
consequence, that is, the need for both institutions and 
governments to manage “change”. 

4.2.5. Concepts of and Approaches to Quality 

At the time of the production of the PHARE Quality Assurance 
Manual (1998), it was observed that all Western European 
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evaluation procedures were based on quality as enhancement 
rather than on quality standards. This landscape, however, is 
a changing one, and work in individual national systems and 
within professional and discipline-related areas at national and 
international level is underway to refine standards, criteria, 
qualification descriptors, and other external reference points 
for quality. While this work stiU includes identification of 
certain quantitative input standards, it is now moving towards 
emphasizing the definition of learning outcomes and output 
standards and competencies in order to create academic 
qualification frameworks that accommodate diversity while 
securing a clear understanding of what academic qualifications 
mean and the associated knowledge and skills students should 
have. The focus is shifting to outcomes rather than process 
(see Annexes 5.1. -5.4. on the Joint Quality Initiative Project, 
the Tuning Project, QAA Subject Benchmark Statements, and 
ZEvA Standards for BAMA Programmes) and on providing clear 
external reference points for quality and standards. The 
organization of quality assurance at system level may be 
perceived as moving (slowly) towards defining “fitness of 
purpose” but with a strong accent on improvement. This latter 
emphasis is borne out by, for example, the articulation of more 
for mal “foUow-up” processes (Sweden, the United Kingdom, 
and the European University Association) to establish what 
improvements had been made following evaluation judgments 
and recommendations. 

The origins of the majority of the external evaluation 
agencies in Western Europe were at the initiative of 
government (see the summary of the INQAAHE questionnaire 
responses in Annex 1). Exceptions include the Netherlands 
and Belgium (Flanders), in which the initiative was taken by 
the associations of higher education institutions, and Germany 
and the United Kingdom, in which initiatives have been taken 
jointly by higher education organizations and government. No 
two national agencies are identical either in aims, focus of 
operations, coverage of the education sector, criteria, and 
standards used. This diversity reflects the different national 
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and regional contexts in which national agencies operate and 
the constituencies which they serve. Even when countries have 
similar social, economic, and political backgrounds, for 
example, the Nordic countries, there is little apparent 
convergence (Smeby and Stensaker, 1999). 

Although established mainly at government initiative, 
evaluation agencies in Western Europe operate as independent, 
autonomous organizations, free of confrol by government or 
universities. This principle was set out in the European 
Council Recommendation on Quality (1998)). In some 
countries, this independence is enshrined in law, e.g., in 
France, where the Comite Nationale d’Evcduation reports to the 
President of the Republic; in others, it is a reflection of the 
nature of the evaluation agency, for instance, in the United 
Kingdom where the QAA is a charitable trust managed by a 
Board of Directors. What autonomy means in practice is that 
the agencies are mandated to carry out approved processes 
and to make their judgments and recommendations without 
interference or influence from government and or higher 
education institutions. This mandate does not mean that 
agencies are not accountable to governments, ministries, or 
other stakeholders or that their processes or indeed their very 
existence may not be subject to change in the light of 
modiflcations in government higher education policy. 

4.2.6. Purposes of Evaluation 

A summary of the responses to the INQAAHE questionnaire by 
members can be found in Annex 1 . It appears to demonstrate 
that the purposes of existing agencies have evolved beyond the 
traditional quality improvement/ enhancement/ accountability 
continuum. Of the fourteen Western European evaluation 
agencies that responded in detaiil, nine (Denmark, Finland, 
Germany (AKK), Norway, the Netherlands (VSNU and HBO- 
Raad), Spain (Catalonia), Sweden, and The United Kingdom) 
indicated three or more purposes as important to very 
important: 

- Quality enhancement was cited by all. 
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- Accountability was cited by eight; however, only most 
important by four (Austria [Akkreditierungsrat - private 
universities <http://www.bmbwk.gv.at>), Belgium (VLIR 
- university QA), the Netherlands (HBO-Rood), and The 
United Kingdom (QAA). 

- Information provision was cited as important by six 
agencies (Denmark, Germany (AKK), the Netherlands 
(HBO-RocicQ, Norway, Spain (Catalonia), and the United 
Kingdom). 

- Benchmarking was rated as important by six agencies 
(Austria Akk), Denmark, Finland, Netherlands (HBO- 
Raadj and (VSNU), and Spain (Catalonia). This criterion 
may be interesting to note, as all are smaller countries 
and/or a regional agency. 

Accreditation as a purpose was mentioned, not unexpectedly, 
by the new, national accreditation agency in Germany 
[Akkreditierungsrat but also as the sole purpose of the 
Portuguese evaluation agency for teacher education (INAFOP 
<http:/ / www.inafop.pt>) and as one of the purposes of the two 
Austrian agencies. In contrast with the Central and Eastern 
European countries, it appears that accreditation as a purpose 
of evaluation is not yet a feature of most European Union 
systems. However, this conclusion belies the fact that in some 
countries, accreditation or homologation of degrees and/or 
institutions is undertaken by the Ministry of Education, e.g., in 
France and Spain (ENQA, 2000, Institutional Evaluations). 
Accreditation-like practices occur in Nordic countries, mostly 
in relation either to professional programmes or to private 
institutions (ENQA, 2001). Accreditation activities of statutory 
and professional bodies as well as the granting of degree- 
awarding powers occur in the United P^ingdom as an ex ante 
type of accreditation for institutions (for criteria see 
<http://www.dfes.gov.uk>). However, the majority approach to 
external quality assurance in Western Europe remains either 
programme and/or institutional evaluation. 

This extension to the purposes of agencies is reflected in 
their aims, tasks, or mission statements and is perhaps an 
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indication that when new agencies are being established they 
should have the flexibility to adapt to changes without the 
necessity to have cumbersome legislation enacted. 

In the case of agencies in the Central and Eastern 
European countries, the purpose of quality assessment is 
accreditation. The responses of agencies to the INQAAHE 
questionnaire were aimed at showing what they perceived as 
their most important purposes beyond the formal judgment of 
compliance with set criteria. The variety of replies very likely 
implies that the question allows for different levels of 
interpretation. Half of the eight Central and Eastern European 
agencies that responded to this question named accountability 
as well as improvement and enhancement as the first or 
second most important purpose of their activity (Albama, 
Hungary, Latvia, and Slovakia). But Bulgaria set 

benchmarking and provision of information first and marked 
accountability least important. For Slovenia, improvement and 
the provision of information were most important, while the 
Czechs focus on benchmarking and accountability. For the 
latter, improvement/ enhancement and providing information 
are tied second. Estonia ranked accountability last, 
improvement orientation second, and benchmarking as 
moderately important, with nothing for “important” or “very 
important” [te„ 4 or 5), and it is possible that the reverse 
ordering was intended. 

4.2.7. Scope of Evaluation: Institutional or Programmatic - 
Universities or Colleges 

All the agencies in the Central and Eastern European 
countries conduct both institutional and programme 
assessment. Usually, the assessments are undertaken in 
separate processes, but Hungaiy, for example, in its first cycle, 
evaluated programmes so as to arrive at an accreditation 
decision regarding the given institution. The Hungarian 
Accreditation Committee is planning on separating the two 
processes, to some degree, in its second round. Croatia focuses 
on institutional assessment but also looks at the quality of 



PROGRAMME DESIGN 



43 



programmes at the evaluated institution. Slovakia seems to 
have introduced the combined approach in its new higher 
education law that went into effect in April 2002. Russia and 
Ukraine also evaluate programmes within their institutional 
accreditation procedure. 

When evaluation agencies were first established in the 
European Union, they tended to focus either on programme or 
on institutional evaluation. The United Kingdom at the 
beginning of the 1990s was unique for a larger state in Europe 
because it covered both institutional and subject evaluation - a 
very expensive operation for a large and diverse higher 
education system. In Ireland, the former NCEA carried out 
validations and approvals at both programme and institutional 
levels. However, agencies in smaller countries or those which 
are regionally based generally began with a focus on 
programme or discipline evaluation on a national/ regional 
basis [e.g., The Netherlands - VSNU and HBO-Raad, Belgium 
(VLIR), Denmark, Spain (regional - Catalonia). 

The evaluation of programmes as a focus has had potential 
benefits both for qualify assurance at system and institutional 
level in: 

- providing an opportunity for small-scale pilot exercises to 
test new processes before their wider implementation; 

- providing an overall picture of the state of health of a 
discipline in a country at a particular time; highlighting 
overall matters for improvement at national level as well 
as in individual institutions; 

- facilitating the planning of a cycle of reviews and 
developing and managing a programme of continuous 
engagement with institutions - which may be necessary 
to build up institutional capacity and familiarity with 
evaluation; 

- potentially creating a multiplier effect by involving a 
wider range of actors as evaluators and creating a 
“critical” mass of actors (academics, administrators, and 
students) in higher education institutions with 
experience of evaluation which may stimulate the 
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development of internal, institutional quality assurance 
mechanisms at “grass-roots level”; 

- where the focus is on discipline or subject, rather than 
programme evaluation, offering the potential of internal 
benchmarking within higher education institutions, in 
which the same subject is offered in several faculties; 

- potentially providing information which is of more direct 
value to students {especially international students) and 
employers; 

- providing comparative information to institutional 
managers and leaders on issues such as teaching and 
academic standards across the institution; 

- providing evidence of the effectiveness of institutional 
quality assurance practices. 

Criticisms have been voiced that focusing on programme 
evaluation may place a considerable burden on institutions as 
they may have to manage multiple evaluation events in any 
one year. These criticisms were aiired most recently in the 
United Kingdom at the conclusion of the first round of reviews 
of all subjects in England. By the conclusion of the eight-year 
period of review, less than 1 percent of provision had been 
judged to be faiiling. However, this result was evidence of the 
effectiveness of internal review processes and contributed to 
the development of a new external review process which will 
utilize information from internal evaluation processes (for 
further details, see Report 02/15, 2002 of the Higher Education 
Funding Council for England <http://www.hefce.ac.uk>) and 
focus, in the main, on institutional audits. 

4.2.8. Institutional Evaluation 

Evaluation of higher education institutions in Europe became 
common as of the mid-1980s. France was the first country to 
initiate comprehensive university evaluations in 1985. The 
Comite National d’Evaluation {<http://www.cne_evaluation.fr>) 
was established by the Law on Higher Education of 1984. The 
Law also granted universities administrative, pedagogical, 
research, and financial autonomy. In addition to the evaluation 
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of individual institutions, the possibility of organizing 
institutional reviews/evaluations based on a regional grouping 
of institutions exists in France. This possibility provides an 
opportunity to determine the extent to which a region is well 
served by its higher education institutions. The second series 
of institutional evaluations in France has a lighter process: 

- Because the “legitimacy” issue was no longer critical, the 
academic community did not need a comprehensive 
report to be assured of institutional unity. 

- Because the CNE wanted its recommendations to 
become more efficient for quality improvement and 
decision-making, only the main issues identified in the 
self-assessment and external evaluation were to be 
included in the recommendations. 

Institutional evaluation is also a focus for quality evaluation 
or audit in Sweden, Finland, Norway, Ireland, and the United 
Kingdom. However, the new Swedish system is evolving from 
quality audit at institutional level to quality assessment of all 
higher educational provision for general and professional 
degrees. A full round of institutional audit was completed in 
1998. The intention of the audit had been to assess the 
strategies, goals, plans, systems, methods, and organization 
used to secure and develop overall quality in each institution 
rather than on evaluating quality against set criteria. 
Criticisms of the process included the fact that institutional 
audit did not reach out within institutions to the core of 
activities at departmental level and that some of the audits 
were considered to be “toothless” (Franke, 2002). 

An alternative to national institutional review is the 
institutional review process offered by the European University 
Association. This opportunity has often been taken up in 
countries that do not have a national evaluation system at 
university level, e.g., Portugal and Greece, or when a given 
institution is interested in adding an international dimension 
to the review (information on EUA quality policy and 
institutional review practices can be found in Annex 4.6 and at 
<http: / /www.unige.ch/eua>). 
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4.2.9. Thematic Reviews and Reports 

Several agencies undertake thematic reports or evaluations 
either at the request of government or other authorities or on 
their own initiative. An example of the latter is the current 
programme of the Danish Evaluation Institute which includes 
evaluations on classroom observation and on methods of self- 
evaluation in the Danish education sector. 

4.2. 1 0. One Agency or Several 

Within binary systems of higher education or when there is a 
mixture of private and state provision, decisions have to be 
taken as to whether there are separate focuses for different 
provision or one process with appropriate standards and 
criteria for different providers. Systems in smaller European 
countries have tended to take an inclusive approach. 
Denmark, Sweden, Finland, and Norway each have a single 
evaluation agency to cover all higher education provision; 
indeed, the Danish agency covers all types of education from 
Kindergcirten to research evaluation. Exceptions are Austria, 
the Netherlands, and Belgium (Flanders). The policy of 
inclusion has also been in effect in the United Kingdom and 
France in which all nationally recognized higher education 
institutions are subject to institutional evaluation by the 
national agency. In the Netherlands, Belgium (Flanders), and 
Ireland, there are separate agencies for university and non- 
university institutions; however, the proposed Dutch 
Accreditation Agency for Higher Education will cover both 
sectors of higher education. In Austria and Portugal, there are 
no evaluation agencies for public universities, but separate 
agencies exist in Austria for private universities and for 
Fachhochschulen, and in Portugal there is an accreditation 
agency for teacher training education (INAFOP). 

The German accreditation system may appear to be 
evolving along the lines of the American system with a 
recognizing body (Akk in Germany, CHEA in the US) which 
accredits agencies operating at regional or professional/ 
disciplinary area. 
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If a trend can be discerned in this aspect of evaluation, it is 
not only one favouring multiple purposes for agencies but also 
multiple focuses of evaluation for single agencies. 

A further development is the existence of multiple purpose 
agencies which have responsibility not only for quality assurance 
or accreditation but also as the authority for the recognition of 
foreign academic and/or professional qualifications. Examples 
include Network Norway Council (<http://www.nnr.no>), the 
Hogskoleverket, Sweden (<http://Www.hsv.se>), and the 
Lithuanian Center for Quality Assessment in Higher Education 
(<http://www.vu.lt>). To some extent, these national initiatives 
have anticipated the growth in networking and contact 
between the emergent QA networks and the academic 
recognition information networks - NARICs (in the European 
Union) and ENICs (for Council of Europe and UNESCO-CEPES). 

4.2.11. Evaluation Criteria and Standards 

What criteria and standards are set for external evaluations? 
Again, there is no one rule for Western European countries, 
and practice varies with approaches which include “an 
enabling legal framework with broad general guidance, re. 
purpose”, to a strict legal framework or one which assigns 
responsibility for determination of standards to the Agency in 
co-operation with the constituencies (or stakeholders) to refine 
reference points. In Finland, the criteria are formulated by the 
agency in consultation with institutions whereas in Belgium, 
(Flanders), they are formulated by the universities. In the 
United Kingdom, the standards are the goals and objectives of 
the given institutions and the external reference points 
developed in agreement with the sector. In Denmark, the 
criteria used are those of the individual institution and 
“general criteria of good practice”. In Germany (AKK), the 
standards are the “go^s as formulated by the institution, the 
profession, and AKK”. 

In response to the INQAAHE questionnaire as to who had 
most influence in setting the criteria for an external evaluation, 
the indications were that the Agency (including for this 
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purpose the external evaluation team) had the most influence 
in determining the focus of individual evaluations. 

4.2.12. Evaluation Methods and Instruments 

A strong consensus exists throughout Western Europe, 
irrespective of process and of the purpose of the external 
evaluation, as to the methods and instruments used. The 
elements common to all systems are: 

- a self evaluation/ study; 

- an external review with an on-site visit; 

- a report; 

and increasingly, the use of other instruments such as 

- statistical data provided either from national sources or 
individual institutions; 

- qualitative information from the internal quality 
assurance processes of the institution itself; 

- performance indicators; 

- user surveys - labour market/students/employers [e.g., 
in Denmark and Finland); 

- external examiner reports (Ireland, The United Kingdom, 
and Denmark); 

- structured follow-up processes. 

Agreement on these common instruments does not, 
however, imply similar implementation practices. There are 
variations in the nature and coverage of the self-evaluation 
reflecting the purpose of evaluation; the criteria and standards 
of evaluation, the aspects of provision under evaluation; the 
duration and structure of visits; the identiflcation, 
appointment, training, and role of experts; and the judgments 
made. Some examples are included in Annex 4. 

4.2.13. The Self-Evaluation Study 

The one common feature about the self-evaluation study is 
that unlike external evaluation reports, self-evaluation 
documents are not published "by any existing evaluation 
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agencies and remain confidential unless the institution 
chooses to release them. While self-assessment is a central 
element of all institutional evaluations, the prescriptiveness of 
the parameters for the self-study as set out by the evaluation 
agency - either in its handbook or through other guidelines - 
varies. Sometimes the self-evaluation document, especially for 
study programmes, will be predicated on legislation and will be 
designed to show how the programme achieves not only the 
aims and purposes established by the institution offering the 
programme but also provisions in appropriate legislation, e.g., 
the FH Council, Austria (see Annex 4.1.1.). In Norway, the 
higher education institution may choose the methodology and 
organizational approach to suit its purposes and interests. The 
position in Finland is similar but with the proviso that the self- 
study reflect the interests of consumers (students, external 
stakeholders, the surrounding community, etc). In Sweden, in 
the new quality assessment process at programme level, 
institutions are encouraged to demonstrate “what is unique 
and especially characteristic about their programmes and by 
so doing highlight... good practice” (France, 2002). 

The British and Irish models have shared a more directive 
approach. In Ireland under the former NCEA, there was 
concentration on organizational matters such as: 

- mission and goals; 

- governance and strategic planning; 

- academic policies; 

- the use of institutional resources; 

all of which were to be reflected in the self-assessment. 

The new institutional audit method which is being 
established in England will, in addition to self-evaluation 
documents at institutional and discipline level, also draw on 
information which all institutions are to be required to produce 
annually from their internal quality assurance processes. An 
excerpt from the Draft Handbook for InstitutiorLcd Audit in 
England (April 2002) - Annex C: “Guidelines for Producing Self- 
Evaluation Documents for Discipline Audit Trails” is 
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reproduced in Annex 4.1.2. The institution being audited is 
also expected to demonstrate the extent to which it addresses 
the points of reference for quality and standards provided by 
parts of the “Code of Practice and the Higher Education 
Qualifications Framework” of the QAA. 

No matter what model of self-evaluation is adopted, there 
must be clear and effective guidance to institutions as to what 
they can or must do. The guidance (which is normally in the 
form of a handbook or other written guidelines and is generally 
publicly available) should assist institutions to undertake 
reflection and analysis rather than directing them to focus on 
description. However, it has been recognized that merely 
providing a handbook or other guidelines may not be sufficient 
preparation for external evaluation, especially when 
institutions have little capacity for, or experience of, 
conducting self-evaluations (Genis, 2002). A programme of 
orientation and capacity building exercises, such as 
workshops on writing self-evaluations and information 
sessions on new QA processes before they are introduced, may 
result in dividends in terms of the quality of self-evaluations 
which will in turn enhance the external ev^uation process. 

4.2.14. External Experts: Selection, Training, and Briefing 

The success of peer review depends heavily on the training, 
experience, and professionalism of teams {KeUs, 1993, p. 104). 
Practices in terms of the training and briefing of teams vary 
considerably in Western Europe, but a common feature is a 
briefing meeting before evaluation visits take place, as are 
guidelines or handbooks on protocol and evaluation 
procedures. The nature of training and briefing to some extent 
depends on the pattern of evaluations. It is easier to coordinate 
training and briefing activities if a pool of experts is selected 
and appointed either for the duration of the cycle or at least for 
a specified period. Practice varies as to who selects the experts. 
In some cases, it is the agency, with either the professional 
staff or appropriate Committee making the decisions; in 
others, there may be nominations from other sources. The use 
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of foreign experience is common in some of the smaller 
countries, particularly when there is a shared language, e.g., 
The Netherlands and Belgium (Flanders) and in the Nordic 
countries, but is also a feature of the activities of the CNE 
(France) and of some of the German Accreditation Agencies. 
Examples of practices regarding the selection and training of 
experts are included in Annex 4.3. 

4.2. 1 5. Reports and Judgments 

In all systems, some kind of public report is generated at the 
conclusion of the evaluation, but the nature and extent of this 
report varies. In those countries in which all provision is 
reviewed at the same time, subject reports may be produced: 
e.g., the VSNU in the Netherlands (see <http://www.vsnu.nl> 
for examples in English). However, the VSNU also produces a 
confidential report for individual institutions on their 
programmes. Other agencies work entirely in the public 
domain and publish reports in full, reflecting their aim to 
inform stakeholders, e.g., QAA (in The United Kingdom) 
<http://www.qaa.ac.uk>. In some countries, reports or 
summaries of reports are published in English even when it is 
not the national language, e.g., Denmark (<http://www.eva.dk>) 
and the Swedish National Agency <http://www.hsv.se>. 
Making reports available on the Internet is a cost-effective 
means of publishing, but alternative means must be available 
to ensure that those who do not have access to the Internet 
can stm obtain the information. 

Practices vary from agency to agency as to who drafts 
evaluation reports. Sometimes it is the agency staff (EVA, 
CNE); in other cases, the agency staff may comprehensively 
edit the report written by the experts but never alter the 
judgments, e.g., QAA. In other cases, the experts may write the 
reports. In the latter cases, the training and briefing of experts 
should take account of the need to assure consistency and 
quality in reporting. 

The nature of judgments also varies depending on the 
purpose of the evaluation. There may be yes/no decisions 
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concerning accreditation and a series of recommendations for 
improvement and/or commendations for good practice for 
evaluation. No Western agency “ranks” programmes or 
institutions, but evidence from published reports, especially 
when they include graded judgments, is often used by others 
to create ranking. Recently, commentators in the United States 
(Jones, 2002) have observed that different stakeholders in 
education are looking for different types of information from 
evaluation processes which present a real challenge for 
evaluation agencies to meet. 

4.2.16. Aspects of Programme Design 

Quality assurance agencies are not responsible for the design 
of individual programmes of study but may influence them 
either through the establishment of external reference points 
or standards of quality or through the recommendations or 
judgments made in the external evaluation of programmes. 

In Western Europe, programme design has ranged from 
being either the sole responsibility of individual institutions 
(British and Irish universities are examples) to the 
responsibility of institutions within a fairly prescriptive legal 
framework which may, however, allow for a certain degree of 
flexibility, e.g., in terms of optional courses in addition to a 
compulsory core curriculum {e.g., Spain). In France, the 
Ministry of Education develops degree programmes leading to 
nation^ awards according to its own guidelines and is in 
charge of the accreditation of degrees and institutions. This 
stipulation does not preclude institutions from designing and 
offering their own programmes, but such programmes do not 
lead to national awards. In other countries, institutions have 
autonomy over programme design and content but are 
provided with reference points such as national descriptors for 
qualifications and programmes either in terms of credits 
(Sweden for academic degrees) and/or study hours and a 
broad subject outline. However, the impact of the Bologna 
Declaration and especially the desire to create greater 
transparency about qualifications for international purposes 
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have set in motion a frenzy of rhetoric, reflection, and activity 
in re-structuring qualifications and higher education provision. 
Much of the activity is incomplete, and there is as yet 
insufficient evidence on which to base judgments as to the 
success of changes in enhancing the transparency of 
qualifications. 

In some systems, the use of credits or credit points - either 
on the basis of a national system, e.g., Sweden, Finland, and 
the Netherlands, or using ECTS (the EXaropean Credit TtBrisfer 
%sban <±ittp://www.euiDpaeu.int/oomm./education>), e.g., Denmark 
and Italy is a requirement in designing programmes. In others, 
use of a credit system is optional. However, credits, per se, 
while a useful tool to organize curricular frameworks, have no 
relationship to quality or standards. 

4.2.17. Two Cycles of Education 

As in Central and Eastern Europe, the Bologna process has set 
in motion a series of changes in relation to the definition of 
qualifications which in turn impinges on programme design. 
The changes include: 

- measures to introduce qualifications based on two cycles 

of education - Bachelor’s/Master’s Degrees - in countries 
where these are “new” qualifications, e.g., Italy, the 
Netherlands, and Germany. Sooner or later, depending 
on the country, these qualifications will replace the 
traditional long, first-degree programmes (See Trends II 
Report for more details at <http://www.oph.fi/ 
publications/trends2>). Where binaiy systems of higher 
education exist, it appears that there will be two types of 
Bachelor’s and Master’s Degree programmes - academic 
and professional fThe Netherlands). An example of 
different descriptors for two orientations of 

Bachelors/Masters degrees in Germany is given in 
Annex 5.2,, re. the standards of ZevA). 

- the redefinition or refinement of qualifications 
frameworks in which two cycles of higher education 
already exist. For example, in the United Kingdom, 
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Higher Education Qualifications Frameworks were 
recently agreed upon and have been established as one 
of the reference points for external evaluation. In Ireland, 
there is currently consultation on a new national 
qualifications framework (<http://www.nqai.ie>. In this 
case, an interesting aspect of the proposal is for the 
inclusion of professional and international awards (that 
is, provision being offered in Ireland but not by Irish 
institutions) in the framework. 

There are two Europe-wide initiatives in the framework of 
the Bologna process which may have an influence on 
programme design and curriculum content through the 
definition of subject specific learning outcomes and the generic 
definition of qualifications (Bachelor’s and Master’s Degrees) 
frameworks: the Tuning project (<http://www.reUnt.deusto.es>) 
and the Joint QuaUty Initiative (<http://www.jointquaUty.com>). 
They are described in Annex 5. 

4.2.18. Emerging Trends in Quality Assurance and the Design 
of Programmes in Western Europe 

- expansion and diversification of the scope and type of 
work that evaluation agencies are carrying out. 
Examples include the estabUshment of agencies with a 
comprehensive range of functions in relation to quality 
and the recognition of qualifications, especially in 
smaller higher education systems (the Nordic countries); 
a greater emphasis on enhancement activities such as 
the introduction of foUow-up processes after evaluation; 
and the undertaking of thematic reviews on wider 
aspects of quality and standards. 

- work on developing instruments to assure and evaluate 
the quaUty of new modes of provision and new providers 
of higher education. This work includes the development 
of codes of practice, both national [e.g., The United 
Kingdom Code of Practice on CoUaborative Provision) and 
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international (the Council of Europe/UNESCO-CEPES - 
<http://www.cepes.ro>), on transnational education, 
the review and updating of evaluation processes to take 
account of the experiences of both institutions and 
evaluation agencies after the completion of a full round 
of reviews. For example, the shift in emphasis from 
institutional audit to programme assessment in Sweden 
and from programme assessment to institutional audit 
in England. 

the development of new external reference points for 
quality and standards such as subject benchmark 
statements and new generic descriptors for BA/MA 
qualifications at regional, national, and international 
level based on student learning outcomes. While this 
work had already started in some countries before 
Bologna, the Declaration has had a catalytic effect as has 
the drive for international recognition of qualifications, 
the introduction or development of accreditation systems 
at national (the Netherlands), regional and subject-based 
(Germany), and international (FIBAA, EQUIS) level, 
an increase in networking in quality assurance and 
enhancement activities: 

* at agency level - the development of international 
networks of quality assurance agencies (ENQA, 
CEENetwork, INQAAHE, CHEA) - all listed in Annex 
6, bilateral links between agencies, e.g., VLIR and 
VSNU, and the exploration of mutual recognition 
between agencies, e.g., a recent project involving EVA 
(Denmark) and FINHEEC (Finland); 

* at institutional level - often on an international basis 
- the emergence of university networks and 
benchmarking clubs, such as Universitas 21 
<http: //www.universitas2 1 .com/collaborative.html> 
and the Coimbra group <http://www.coimbra- 
group.be>; 

* at subject level, e.g., Socrates thematic networks; 
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’ by the professions, e.g., the mutual recognition 
activities in engineering by accreditation agencies 
through the Washington Accord (See, Annex 4.7.2. for 
information). 

- greater involvement of stakeholders, especially students, 
in the external evaluation process. Examples of this 
involvement include the participation of the European 
Union of Students (ESIB <http://www.esib.org>) as 
members of ENQA, student representation either on the 
councils of agencies [e.g., in the United Kingdom) or on 
visiting panels (Sweden). But, this involvement is more 
than “representational”. Recent revisions of external 
evaluation have put the interests of students at their 
core. “Students are central both to the principal focuses 
of institutional audit and to the audit process itself. 
Audit teams scrutinize a range of matters directly 
relevant to students, including the quality of the 
information provided for them, the ways in which their 
learning is facilitated and supported, and the academic 
standards they are expected to achieve, and achieve in 
practice. In each audit, students are invited to 
participate in the key stages of the process....” (QAA, 
2002). “The student perspective is also very important. 
They constitute one of the most important groups that 
the peer group team meets at visits and are always 
represented on the expert groups as well” (Franke, 
2002 ); 

- the provision of more and better information on quality 
and standards which is prompting: 

• debates about changes in the nature of judgments 
made by agencies; whether or how to make 
judgments without ranking. For many existing 
accreditation systems, such a shift would imply 
changing from summative to formative judgments; 

• debates about disclosure or wider publication of 
evaluation reports. In many easting systems, 
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“publication of reports” means only sending them to 
the institution that has been evaluated or making a 
summary of the report available more generally. 
Wider disclosure need not mean enormous 
publication costs but does imply the setting-up and 
maintenance of efficient Website and negotiations 
with institutions. The degree of expense has 
implications for both national systems and 
institutions. International students are becoming 
increasingly sophisticated in searching for 
information about study opportunities and frequently 
use the Internet to do so. Wider publication of reports 
may also provide an opportunity for the spreading of 
information about good practice, thus contributing to 
the enhancement aim of external evaluation; 

• the need to improve the “language” about quality - for 
many agencies, the public is the ultimate constituent, 
and much of the language of quality and standards is 
jargon-ridden and not immediately useful to 
constituents such as government or students, 
(examples include publication by QAA of leaflets for 
students; the CHEA glossary of terms), 
closer links at national and international level between 
qualify assurance agencies and bodies responsible for 
the recognition of qualifications for academic and 
professional purposes. At international level, an example 
of such a linkage is the co-operation between ENQA and 
the ENIC and NARIC networks (more information about 
ENICs from <http://www.support41eaming.org.uk/ 
education/narics.htm>). The outcomes of two meetings 
between representatives of the networks was the 
identification of specific issues for further work: 
improving the communications between the network 
members both at national/regional level as well as at 
network level and how to improve the definition of 
qualify and recognition issues in non-formal education. 



58 C. CAMPBELL and C. ROZSNYAI 



- the emergence of other “non-formal” commentators on 
quality and standards who “rank” institutions and 
programmes. Examples of this phenomenon are the 
rankings of universities by newspapers such as Der 
Speigel (Germany), the Financial Times (business schools 
around the world), The Guardian (British universities), 
etc., even when national quality assurance agencies 
themselves make no rankings. While this trend may not 
be viewed as positive, it does, nevertheless, demonstrate 
an interest in and demand for information on quality 
and standards. 

However, “a national evaluation system can only best be a 
supporter of the activities taking place in the educational 
process” (Franke, 2002), and there is a growing awareness of 
the need to strengthen internal quality assurance and 
management in higher education institutions in many Western 
European countries. There is still a perception in many 
institutions that quality assurance is an external activity or 
control imposed on them. A recent initiative in response to this 
perception is a proposed project, to be managed by the 
European University Association, to focus on the development 
of internal quality cultures in European Universities. 

This initiative also brings into focus an issue which is not 
about quality management per se, but is closely related to it, 
te., the perpetuation in some Western European countries of 
rigid governance and outdated management structures for 
higher education which mean that tertiaiy education 
institutions are not themselves in a position to exercise 
effective control over the principal factors affecting the quality 
and costs of their own programmes (World Bank, 2002). 
Matters such as granting institutions more freedom and 
authority over curricula, teaching methods, the appointment 
and promotion of staff, and the selection of students, as well as 
over the improvement of leadership and management are also 
essential in order to create institutions which will be able to 
deliver on the targets set by the Ministers of Education at 
Bologna and Prague. 
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4.3. QUALITY ASSURANCE AND PROGRAMME DESIGN IN 
THE CENTRAL AND EASTERN EUROPEAN COUNTRIES: 

THE EVOLUTION AND CURRENT STATE OF AFFAIRS - 
DESCRIPTION AND COMPARATIVE ANALYSIS 

After the fall of communism in and after 1989-1990, all the 
Central and Eastern European countries began to restructure 
their higher education systems. Acts on higher education were 
introduced which gave tertiary education an autonomy it had 
not known for half a century or at all, allowing the respective 
systems to set up internal decision-making bodies 
(institutional and faculty senates, research committees, etc.), 
giving them hiring and firing powers, and involving students in 
institutional governance. At the same time, the acts and other 
legislation installed a number of control mechanisms. 

The hiring and firing of academic and administrative staff, 
who are public sector {te., government) employees, for 
example, is also regulated by public employment laws. The 
legislation, moreover, has outlined the rniriimum requirements 
for a higher education institution with regard to its functioning, 
structure and organization, finance and governance, activities 
and services, and has usually defined the t 3 rpes of institutions 
that may operate in a given country. The laws describe the 
requirements for study programmes with regard to structure 
and minimum content as well as to different levels of 
education. In many countries (Bulgaria, Estonia, Hungary, 
Latvia, Lithuania, Romania, Russia, Slovenia, and Ukraine), 
the laws stipulate the promulgation of national qualification 
requirements or subject benchmarks which establish the 
general content, examination schemes, and criteria for degree 
programmes. 

In designing their study programmes, higher education 
institutions have thus had a high degree of autonomy to 
accommodate the needs of their stakeholders and 
environments. At the same time, they are bound by regulations 
as to both the way their programmes are made up and ex ante 
and/or ex post quality checks. 
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The establishment of democracy in the Central and Eastern 
European countries released tremendous energies and affected 
all walks of life, including higher education. The time frame 
coincided with such global trends in higher education as the 
sharp increase in the demand for study places, the emergence 
of a large variety and flexibility of study programmes and 
schemes, the entry of information technology into the market 
for educational provision, the increasing interest in quality 
assurance and government demands for accountability in the 
use of public monies, and calls for stakeholder protection. 

Different movements were set in motion that gained 
momentum in Europe through various European Union 
programmes {e.g., TEMPUS, SOCRATES-ERASMUS, PHARE, 
NARIC), the Council of Europe, and UNESCO/CEPES (e.p., the 
Legislative Reform Programme, ENIC), OECD, and others. The 
Bologna Declaration is perhaps the most far-reaching of all 
European initiatives, which in turn has set into motion various 
movements to design the future of European higher education. 

The Central and Eastern European countries have had to 
face the challenge of taking advantage of the opportunities 
offered by the various trends and to adapt them to their own 
historical, political, cultural, and social environments. Fifteen 
Central and Eastern European countries, all of them, with the 
exception of Serbia/Montenegro, have, in place, some sort of 
national-level quality assurance body for higher education. TTie 
mechanisms they assign to assessing quality are similar. TTie 
terminology is largely the same. TTie implementation and its 
ramifications are both similar and different in each country. 

TTie question as to why all the Central and Eastern 
European countries chose accreditation as their mechanism of 
quality control in higher education, while the Western 
European model in the early 1990s was assessment/ 
evaluation-oriented, has been analysed by various authors 
(Kristoffersen et al., 1998, pp. 21-22; Tomusk, 2000, pp. 175- 
185). The reasons cited include the need, with the emergence 
of democracy, to establish comparability with Western higher 
education; the necessity to re-evaluate the curricula to rid 
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them of politically distorted content; and the urgency to 
modernize programme content and approach as well as to 
introduce more flexible programme structures. “Before 1990, 
higher education was still an ‘elite’ sector with rigid entrance 
requirements and, in many places, semi-autonomous faculties. 
Everyone knew the “rank” of a college or university;... new 
rules of survival in the institutional hierarchy, a brain-drain... 
appeared” (Rozsnyai, 2001, p. 3). Answers, such as a steep rise 
in private institutions in some countries and in pressure to 
allow access for a large number of students to a previously 
elite sector, may also be part of the explanation; however, 
these trends were experienced in Western Europe as well, 
albeit more gradually. 

There has been the blunt postulation that accreditation is 
the approach most suited to a region accustomed to an 
autocratic mentality (Tomusk, 2000, pp. 175-185). It appears 
that all of the above have been factors in the preference of the 
Central and Eastern European countries for accreditation over 
evaluation or assessment. 

The systems of no two countries. East or West, North or 
South, are alike. All have evolved according to a handful of 
models, but have adapted these models to suit their own 
historical and cultural contexts. The fact that several Western 
countries are introducing accreditation as a form of quality 
control and accountability does not mean that the approaches 
there are the same as those in the Central and Eastern 
European countries, nor are they the same in any two Western 
countries. The fact that the political regimes changed - utterly 
- in the Central and Eastern European countries almost at the 
same time may also explain why almost all opted for 
accreditation. 

The threat to close down institutions, which is inherent in 
accreditation, has rarely been used in the Central and Eastern 
European countries (even though more so where state 
accreditation is viewed as a tool to control the extensive 
proliferation of new private institutions, e.g., Romania and 
Russia). Various forms of warning, giving institutions or 
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programmes time to improve, were implemented everywhere. 
In fact, a trend can be observed that quality assurance 
agencies and committees in almost all countries are 
increasingly stressing the importance of an improvement 
orientation and relying more on the internal quality control 
mechanisms in higher education institutions. 

An interesting explanation for the function of accreditation 
was given for Poland, but it holds true for most of the Central 
and Eastern European countries. 

The reason why accreditation is proving itself so popular in 
Poland is that, like elsewhere in Central and Eastern Europe, it 
is probably the only known mechanism in the world with an 
element of certification, a classification of institutions into 
those meeting the set standards and those failing to do so.... 
On the other hand, the accreditation committees are not 
merely stating whether [or not] the various curricula meet the 
minimum requirements, but they also look at the degree to 
which these requirements are exceeded and compare this 
“excess” against the achievement of set goals (fitness for 
purpose). In this sense, the work of accreditation committees 
has got more to do with the kind of quality assessment carried 
out in many Western European countries, with accreditation 
being merely a form of utilizing the results of this assessment. 
The name, “accreditation committee”, is thus misleading, with 
“assessment and accreditation committee” being a more apt 
designation (Wojcicka and Chmielecka, 2001). 

The INQAAHE questionnaire was filled out by almost all the 
Central and Eastern Europe Network member countries. The 
responses show that in Albania, Bulgaria, Estonia, Hungary, 
Lithuania, and Romania, the initiative to establish the agencies 
came from the government (or from Parliament). The same is 
clearly true for Russia and Ukraine, which did not submit 
questionnaires. In all these countries, the government also 
holds ownership of the agency, except in Estonia (where it is a 
foundation). (Latvia marked both government and higher 
education institutions as holding ownership, while the Czechs 
said that their agency was an independent body. Both 
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presumably had an unclear understanding of the question - 
similar to Macedonia which indicated that both the 
government and higher education institutions established the 
agency. In these three cases, the government was the main 
driver, while clearly the higher education institutions were 
involved to some degree in most countries). In Macedonia, the 
ownership of the agency is an inter-university conference. 
Poland set up agencies initiated by higher education 
institutions in addition to the government one. The initiative in 
Slovenia also came from higher education institutions, but 
these indicated that the government holds ownership. 

To understand the questionnaire correctly, however, it must 
be noted that many Central and Eastern European countries 
distinguish between “agency” and “council” (or “committee” or 
“commission”). The “agency” is understood as the office in 
charge of coordinating evaluations, which may mean an)d;hing 
from mere administrative activities (e.q., the Czech Republic) to 
actually being involved with evaluation teams. The commission 
is the body of experts which approves accreditation decisions 
which may be based on the recommendations passed on by 
the agency. The two must be taken as a unit and, in fact, 
many countries responded with this understanding, even 
though the “agency” may play a quite active role in the process 
{e.g„ Hungary). 

The degree of independence of quality assurance agencies 
or committees vis-a-vis the governments of their countries 
varies. In some countries, higher education institutions have 
taken the initiative to establish quality assurance bodies 
(Macedonia, the Polish agencies, the Slovenian QAC), but the 
governments in all of the Central and Eastern European 
countries have some control over quality assurance. Some 
agencies operate within their ministries (the Czech Republic, 
the Polish State AC, Russia, the Slovenian Council, Ukraine) or 
are situated, more or less loosely, under ministerial 
supervision (Croatia, the Estonian Council, the Latvian 
Accreditation Commission, Hungaiy, Lithuania, Slovakia), or 
under that of the respective Council of Ministers (Albania, 
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Bulgaria) or the parliament (the Latvian Council, Romania). 
The various jurisdictional arrangements are not a direct 
indication as to the degree of independence. The committees 
making decisions as to the quality of institutions, programmes, 
or other higher education issues are almost always guaranteed 
autonomy by law. In Hungary, the Minister of Education may 
take a decision opposed to the opinion of the Accreditation 
Committee only if he or she publishes his or her reasons for 
doing so. In the Czech Republic, the decision of the 
Accreditation Commission is even more binding. If it 
recommends not accrediting a programme, the minister may 
not override its judgement, and if the Commission adopts a 
positive decision, the Minister may refuse accreditation only on 
explicitly stated grounds. Likewise, in Slovakia, the Minister 
must publish dissenting decisions, and in Estonia, he or she 
may modify a decision made by the council or the commission 
only on legal grounds. In Romania, the Parliament or 
Government adopts decisions relative to the accreditation of 
institutions and programmes, respectively, and does not, as a 
rule, diverge from the decisions taken by the commission, even 
though it potentially could. In Bulgaria, Croatia, Estonia 
(Council), Lithuania, Macedonia, and Slovenia (National 
Commission), on the other hand, assessment or accreditation 
decisions are advisory and not binding on the Minister. In 
Albania and Macedonia, moreover, one or more ministry or 
government delegates participate in commission meetings. In 
Russia and Ukraine, the accreditation body is located in the 
Ministry; however, particularly in Russia, an agency assists in 
the procedure (and some forms of education are accredited by 
local /regional authorities). 

All commissions work with external pools of experts. 
Estonia recruits its experts from foreign countries, in part by 
approaching accreditation agencies to recommend them. The 
experts are scholars and researchers in the fields to be 
evaluated. They remain in Estonia for a week during which 
they visit all institutions teaching in the field and write their 
reports. The Hungarian Accreditation Committee, on the other 
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hand, selects peer reviewers on the recommendation of 
committee members and those of its subcommittees dealing 
with issues in individual study fields. Foreign reviewers are 
invited in fields in which there are no or only a limited number 
of experts in the country. 

Higher education institutions in most countries have some 
form of internal quality assurance mechanisms. The provision 
is stipulated in the Higher Education Act in the Czech 
Republic, and higher education institutions must publish the 
outcomes. In Hungary, the Higher Education Law decreed that 
institutions must set up quality audit commissions by mid- 
2000. The Slovenian Quality Assessment Commission was set 
up in order to assist institutions in implementing self- 
evaluation. Bulgaria has just completed a PHARE project to 
introduce internal evaluation at higher education institutions 
and may, in the future, revise its external assessment to 
include the internal results. The new higher education law in 
Macedonia lays great emphasis on institutional-level quality 
assurance. Certainly, self-assessment is part of the quality 
control process in aU countries. 

The procedures for evaluation/ accreditation in the Central 
and Eastern European countries are in line with the 
predominant model, te., they employ self-evaluations, external 
reviews with on-site visits, and written reports. The content of 
and approach to self-evaluation vary from mostly in-put 
oriented to analytical results {Croatia, Estonia, Latvia, and 
Lithuania are examples for the latter). The focus of the 
evaluation/ accreditation reports by the commissions also 
varies. All countries (no data available for Ukraine) stress the 
improvement orientation of their quality assurance systems, 
even though the degree and mode of achievement of this aim is 
interpreted differently within the various cultural environments. 
Reports may be published (Albania, Czech Republic, Hungary 
after 2002, Latvia, Macedonia); however, in some countries, 
whether or not publication occurs is conditional on permission 
being granted by the individual higher education institution 
(Romania). In other countries, reports are intended for internal 
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institutional and/or government use (Bulgaria, Croatia, 
Estonia, Hungary until 2001, Lithuania, Poland, Russia, 
Slovakia, the Slovenian Commission, and Ukraine), while the 
final decision and a brief explanation may be made publicly 
(Bulgaria, Hungary until 2001, Lithuania, the Slovenian 
Council, Ukraine). 

In what follows the conditions for programme design in 
various Central and Eastern European countries are 
described, followed by an overview of their approaches to 
quality assurance. Annex 2 offers a country-by-country review 
of higher education in fifteen Central and Eastern European 
countries with a focus on their quality assurance systems; and 
a subsequent set of tables provides an overview of 
accreditation/quality assurance agencies, but first, a brief 
description of the methodology used to compile this 
information. 

4.3.1. Methodology 

Much of the information concerning quality assessment and 
accreditation in the fifteen Central and Eastern European 
countries studied was gathered by means of a questionnaire 
when the Central and Eastern European Network of Quality 
Assurance Agencies was established in 2000. This information, 
that is available on the Network Website, is updated regularly, 
the updating being based on information supplied by the 
country agencies. Most Central and Eastern European 
agencies have written short reports for the Central and Eastern 
European Network Website. These offer a brief description of 
the higher education of the countries and an overview of their 
quality assurance systems. A great deal of additional 
information was provided very generously and in great detail 
by contact persons in the respective countries, at quality 
agencies, research centers, and ministries. A set of questions 
was compiled for this purpose but answers were not limited to 
these. An exception is Ukraine, in which no contact could be 
established and which has not joined the Network. The 
information on Ukraine given in this publication comes mainly 
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from a study published by the German Rectors’ Conference 
(see Part III). 

After the tables and country profiles on higher education 
and quality assurance systems had been completed, they were 
sent to the contact persons in the respective countries for 
checking. The replies made by these persons are incorporated 
in the pertaining sections. 

One set of listings in Section 2.1 of the Bibliographies in 
Part III provides descriptions of the higher education and 
quality assessment systems of certain countries. Another set 
concerns more than one country or general topics related to 
quality assurance and higher education. Much, but not all, of 
the listed literature was provided by the contact persons of the 
quality assurance agencies that are part of the Central and 
Eastern European Network. 

4.3.2. Programme Design 

Common to all higher education systems in Central and 
Eastern Europe is that the tertiary study programmes they 
offer: 



- are defined in national legislation; 

- are comparable within a given country and, for the most 
part, are compatible with those in other countries; 

- issue closing documents which certify qualification in a 
given field and indicate the level of education achieved, 
and enable persons who have successfully completed the 
requirements to seek employment based on such 
documents; 

- are subject to quality control. 

Before opening a study programme, higher education 
institutions in most countries must submit applications for 
licensing or accreditation to their national quality assessment 
agencies or commissions (Albania, Bulgaria, the Czech 
Republic, Hungary, Latvia, Macedonia, Romania, Slovakia, and 
Slovenia). Some systems are very flexible with regard to joint 
programmes among different institutions [e.g., Slovenia), while 
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in others [e.g., Hungary) such initiatives are less common, 
even though they exist. 

4.3.3. Levels of Study 

The stipulation in the Bologna Declaration that all higher 
education systems in Europe offer “a system of easily readable 
and comparable degrees [whereby] access to the second cycle 
shall require successful completion of the first cycle studies....” 
(Joint Declaration..., Bologna, 18 June 1999 <http://www.mab.hu/ 
eriglish/a_liriks.html>) has created a bustle in some countries in 
which the two-level or -tier (Bachelor’s /Master’s Degree) 
system was not applied. They have had to search for models in 
order to restructure their tertiary provision. The challenge is 
more pronounced in those countries with binary higher 
education systems in which higher education and degrees are 
offered in two distinct types of institution, either at universities 
or at non-university establishments. 

Most prominently Germany, the home of the “Humboldtian 
university model” (pursuant to one of Wilhelm von Humboldt’s 
main principles behind his university reform, the “unity of 
teaching and research”), has traditionally regarded universities 
as educational institutions with a strong link to (theoretical 
and methodical) research, while the relatively newer 
institutional types of the country, the Gesamthochschulen 
(comprehensive universities) and especially the Fachhoch- 
schiden (universities of applied sciences), were introduced to 
expand the provision toward more applied education and 
training. The binary system was introduced relatively late in 
Germany as compared to countries that had relinquished the 
unitary system much earlier in their - modem - history or had 
a more diverse stmcture essentially from the beginning. There 
was no provision for changing from one type of institution to 
another. 

Under Soviet dominance, many CEE countries, which 
earlier had followed the German model, installed the Soviet- 
type binary system with so-called specialized higher education 
institutions, which were of the college level with an applied 
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orientation in a particular general field. Under the Soviet 
model, pure research was assigned to the national academies 
of sciences and their institutes. Universities provided 
theoretically-oriented education while non-university institutions 
or colleges offered applied education. The possibility for 
students to move between the two types of institution was not 
bufit into the system. The systems of Albania, the Czech 
Republic, Hungary, Romania, Slovakia, Russia, and Ukraine 
are of this type, as was Macedonia until the Higher Education 
Law of 2000 made provision for non-university professional 
institutions. Whereas under the Soviet model, doctorates were 
awarded by the academies of sciences, after 1990, all countries 
have (re)introduced Doctoral studies at universities. 

Recently, a number of countries have adopted the two-level 
system of studies. In recognition of the importance of 
comparability, mobility, and the message of the Bologna 
Declaration, as well as the will to be part of the European 
higher education area, the sector is groping for ways to 
establish two-level study systems. While so-called Bachelor’s 
Degree and Master’s Degree programmes are on offer in many 
countries, they often mean only that the former are short, 
usually practical-type programmes, and the latter, longer, 
research-oriented programmes, sometimes at different types of 
institutions. “Master’s” studies may also designate two or 
three-year programmes offered for holders of Bachelor’s 
Degrees and also one-stream, five or six-year programmes, 
depending on the field (usually medicine, some engineering 
programmes, and some social sciences or humanities 
programmes). Doctoral studies, offered at universities and 
often in collaboration with external research institutes, in fact, 
constitute the third level of higher education stipulated in the 
Bologna Declaration. 

With respect to institutional types, the two-level higher 
education system also exists in Albania, but only the nursing 
school is non-university-level to date. At the same time, 
universities already offer many different levels and diplomas or 
degrees, with one to two-year Master’s Degree level studies 
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built on Bachelor’s Degree qualifications, attained after four to 
six years, followed by Doctoral studies. The implementation of a 
credit transfer system is impending. 

In Bulgaria, the binary system predominates, with separate 
streams for Bachelor’s and Master’s Degree level studies. 
However, in some fields it is possible to earn a Master’s Degree 
after one year of study after completion of a Bachelor’s Degree 
(4 years). The Bulgarian higher education system includes the 
educational qualification degree, “Specialist in...’’, a degree of 
professional higher education attained after at least three 
years of studies at a college; the Bachelor’s Degree (first degree) 
attained after at least four years of studies in a university or a 
higher school; the Master’s Degree (second degree) attained 
after at least five years of training or at least one year after the 
Bachelor’s Degree in a university. The training for the Doctoral 
Degree is provided along research specialties following an 
earned Master’s Degree in a university or in the Bulgarian 
Academy of Sciences, the Agriculture Academy, or other 
research organizations. 

For its polytechnics, Croatia has two-level higher education 
programmes offered at the undergraduate level (2-4 years) and 
at the graduate level (1-2 years). University degrees can be 
earned after four to five years; Science Master’s Degrees, after 
six to seven years, altogether. 

In the higher education system in the Czech Republic, 
Bachelor’s Degree programmes require three to four years to 
complete, and most Master’s Degree programmes require five 
or five-and-a-half years. It is also clear from the fact that in 
1999, 71 percent of the programmes offered were Master’s 
Degree programmes that these are one-level studies. A variety 
of choices are evident; however, most higher education 
institutions offer Bachelor’s Degree Programmes (however, not 
in certain fields, such as medicine or law). Nevertheless, two or 
three-year Master’s Degree programmes are accessible to 
holders of Bachelor’s Degree diplomas. The Czech Parliament 
is elaborating a law to introduce the two-level system across 
the board. 
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The government of Estonia issued a regulation in June 
2000, the “Higher Education Standard”, which determines 
Bachelor’s Degree (3-5 years) and Master’s Degree (1-2 years) 
studies. Admittance to the latter is based on successful 
completion of the former. The regulation sets down the 
minimum requirements for the two levels, as well as the lower, 
vocational (“diploma”, three to four years), and the higher. 
Doctoral level. Some fields, such as human and veterinary 
medicine or architecture, are covered in one stream of five to 
six years. 

In Hungary, a clear division is made between “colleges” and 
“universities”. The Higher Education Act defines college 
degrees as comparable to Bachelor’s degrees and university 
degrees as comparable to Master’s Degrees (Hungarian Higher 
Education Act LXXX/1993 [last amended in 2000], Section 24 
§ 95 (6)a) and b). So-called “complementary undergraduate” 
programmes, stipulated in the 1996 Amendment to the Higher 
Education Act, are university (or teacher training) programmes 
for college graduates. Yet with the exception of a limited 
number of bilateral agreements between specific universities to 
take on graduates from specific colleges, it is very difficult for a 
college graduate to enroll in university studies at a comparable 
year level. Various options for the introduction of two-level 
studies are being discussed. A preference is emerging for 
running both models together, which would retain full, five or 
six-year programmes in certain study fields [e.g., medicine), 
and change the curriculum of others [e.g., computer science) 
to two-level programmes. Hungary also has two-year 
vocational higher education programmes, one third of which 
may count toward college or university studies. 

The Latvian Law on Higher Education Establishments lays 
down the three levels of higher education studies: Bachelor’s 
Degrees, Master’s Degrees, and Doctoral Degrees. The duration 
of a Bachelor’s Degree programme may not be less than four 
years, while a Master’s Degree can be attained after five to 
seven years, altogether. Some fields offer Master’s-level studies 
in one stream. 
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In 1993, Lithuania introduced Bachelor’s Degree programmes, 
which, as of 2002, are extended from three-and-a-half to four- 
and-a-half years, and one-and-a-half or two-year Master’s 
Degree programmes, while some fields, such as medical 
studies, continue as six-year programmes. There are also 
vocational one- or two-year programmes. Ordinarily, in 
university- type institutions, there are three levels of studies: (i) 
undergraduate, leading to a Bachelor’s Degree and/or 
professional qualification; (ii) graduate, leading to a Master’s 
Degree and/or a professional qualification; and (iii) postgraduate, 
leading to a Doctor’s Degree or residency (for medical doctors) 
or “meno aspirantura” for artists. In non-university type 
institutions, studies lead only to professional qualifications. 

Macedonia also has the three levels of study: Bachelor’s 
Degree, Master’s Degree, and Doctoral Degree. Bachelor’s 
Degrees [diplomiran) are issued after four to six years. The 
longer studies include fields such as medicine and are actually 
of the German-type university level. Master’s Degrees 
{magister) are awarded after two years of study following a 
Bachelor’s Degree. 

Poland, while having a two-level Bachelor’s Degree {licencjat 
or inzynief) and a Master’s Degree system, retains one-level 
studies in certain fields. Universities offer professionally 
oriented programmes of between three and four years after 
which a two- to two-and-a-half years Master’s Degree 
programmes may be entered. In fields such as law, pharmacy, 
or medicine, there is one stream of studies of five to six years. 

Romania is undergoing a fundamental structural reform of 
education, begun in 1997. It soon introduced Master’s Degree 
programmes, understood as post-graduate studies after a 
university degree, which is considered a Bachelor’s Degree. 
The reforms also extended to the restructuring of PhD 
programmes. Traditionally, short, three-year programmes have 
been offered in some fields at university colleges, which lead to 
a university-college diploma. Graduates with a university- 
college diploma may take entrance examinations to enter the 
third year of long programmes. Licentd diplomas are degrees 
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awarded in certain fields after four to six years, at universities, 
academies, and conservatories. Graduates with licenta 
diplomas may continue their studies in one-year specialized 
programmes or one- or two-year Master’s Degree programmes. 
One-year Master’s Degree study may count toward study for 
the PhD degree. 

Russian higher education may be of the non-university 
level, not leading to an academic degree, and of university 
level, leading to an academic degree. The former consists of 
“technical schools” (that offer courses in the humanities, the 
social and natural sciences) following three years of general 
education and two to three years following secondary 
education and colleges offering course programme running 
three-and-a-half to four-and-a-half years. The latter are offered 
at universities, academies (for one area of study), and 
institutes (independent or part of a university or academy, for 
post-graduate studies in any field). Russia has retained several 
levels of Doctoral degrees. These are not conferred by the 
Academy of Sciences but by the Academic Councils of higher 
education institutions. There is no course work, only 
independent research with a consultant followed by a 
dissertation and its defense. 

The one-level system is also prevalent in Slovakia, with 
what is considered Master’s Degree-level diplomas representing 
as much as 95 percent of the graduates to date. At the same 
time. Master’s Degree studies that are based on Bachelor’s 
Degrees, as well as programmes leading to independent 
Bachelor’s Degrees, are gradually being introduced. The new 
Higher Education Act, in effect since 1 April 2002, defines 
Bachelor’s, Master’s, Engineering, and Doctoral levels. 
Bachelor’s degrees are earned in three to four years; Master’s 
Degrees generally require an additional two years, but 
depending on the field, programmes may be offered only at the 
Master’s Degree level in four to six years. Engineers receive the 
title of “engineer in...” the particular field. Medical studies lead 
to the doctor’s title with the speciality given. Both are “second 
grade”, le., Master’s-level studies. TTie third level is the PhD 
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and other Doctor’s Degrees, usually requiring three years to 
complete. 

Slovenia, on the other hand, already had a two-level system 
in 1995 (The Higher Education Act, in, Zgaga and Jurkovic, 
1995) in its first higher education law (before Bologna). In fact, 
a statistic shows that “MSc”-designated programmes have 
been offered in this country since 1970 (Zgaga and Jurkovic, 
1995). Bachelor’s Degree-level programmes are in part offered 
by faculties of universities that were originally junior colleges 
that awarded professional diplomas after two years of study. 
Following their integration into the universities in 1995, they 
offer three-year studies. 

Higher education in Ukraine has four accreditation levels: (i) 
junior specialist courses of a maximum of three years of 
duration (at vocational and technical schools); (U) four-year 
Bachelor’s Degree courses offered at colleges; (Ui) specialist 
degree courses lasting an additional two to three semesters 
after a Bachelor’s Degree, or five to six years as full 
programmes offered at institutes, conservatories, academies, 
and universities; (iu) Master’s Degree programmes, requiring 
the defense of a research thesis, of six years for secondary- 
school graduates, of two to three years for those with junior 
specialist diplomas, or of one to two years on top of Specialist 
or Bachelor’s Degrees offered at institutes, conservatories, 
academies, and universities. In addition, there are three-to- 
four-year Doctoral programmes leading to “Candidate of 
Science” Degrees. The subsequent “Doctor of Science” Degrees 
are offered after completion of a dissertation following 
independent research and sometimes course work and are 
conferred by a High Certification Committee. 

4.3.4. The Legal Setting for Programme Design 

Quality assurance agencies are not directly involved in 
developing study programmes. Assessment does, however, 
check compliance with the legislative requirements regarding 
programmes. Indirectly, moreover, the assessment has an 
influence not only on the quality of programmes but their 
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content as well, in that the evaluation report points out the 
strengths and weaknesses of a programme. 

All of the Central and Eastern European countries have 
passed legislation governing higher education. This legislation 
describes, to varying degrees, the requirements and standards 
for study programmes, which is the condition for state 
recognition of programmes. Within the boundaries of the legal 
framework, the higher education institutions in all countries 
are free to design their own study programmes. Before 
launching them, however, the programmes must generally be 
passed by the respective accreditation body and approved by 
the Ministry or Higher Education Council. 

In Albania, the Ministry of Education and Science is 
responsible for approving new educational programmes. 
Institutions are redesigning their programmes and structures 
according to standards approved by the ministry. 

In Bulgaria, new programmes must be approved by the 
Accreditation Council before they can be launched. In addition, 
various sections of the Higher Education Act describe the 
requirements for programmes, including examination 
procedures and the educational goals of programmes. Within 
this framework, institutions design their own programmes in 
accordance with their internal regulations. 

In Estonia, institutions design their own study programmes 
in compliance with the general legal framework. A government 
regulation of June 2000 governs the levels of study and of 
study fields in eight categories. It also elaborates rninimum 
duration, content, and staff qualifications as well as 
examinations. 

In the Czech Republic, the Ministry of Labour monitors the 
unemployment distribution of graduates so as to identify study 
fields in which prospects for finding work are positive or 
negative. Higher education institutions also conduct internal 
evaluation of their activities, including their programmes. 

In Hungary, a government decree has grouped fifty- six 
disciplines into eight areas of science and the humanities 
under which all study programmes are categorized. This 
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categorization has importance in that universities and colleges 
are defined by having the capacity to teach in a set number of 
areas, and Doctoral schools, for example, may be launched if 
the university is accredited in the discipline to which the 
Doctoral school is assigned. Another set of government decrees 
concerns national qualification requirements. They set down 
the standards for the general content of a study programme, 
the examination schedule, and, in general, the knowledge to be 
transmitted by means of it. Qualification requirements for new 
study programmes or programmes to be launched at a higher 
education institution are submitted to the Hungarian 
Accreditation Committee for a quality check. 

In Latvia, institutions also design their own programmes, 
but separate legislation governs education, higher education 
establishments, and the general principles of accreditation, 
and hence programmes standards. The 1998 Education Law 
and the Law on Higher Education Establishments provide the 
broad framework within which higher education institutions 
may develop their study programmes for aU types and levels. A 
licensing requirement must be met before programmes can be 
launched. 

The Lithuanian Law on Higher Education stipulates the 
development of subject area benchmarks describing the 
requirements for educational programmes, based on which 
higher education institutions develop their programmes and 
curricula. The studies in higher education institutions are 
based on the programmes included in the State Registry of 
Studies and Programmes, which was established in 1997. 
Studies are organized in accordance with the curricula 
prepared by institutions of higher education which abide by 
the regulations. 

In Macedonia, the Accreditation (also translated as the 
“Licensing”) Board licenses new study programmes, which are 
developed by higher education institutions. The legislative 
framework for study programmes is laid down in the Higher 
Education Act. 
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Romania, in line with its education reform launched in 
1997, revised its list of university specialization. Recent 
legislation, including two government decrees in 2000, assert 
the right of universities to develop their own curricula. 
Curricular standards are regulated by a ministerial ordinance 
(1998). 

Russia has state educational standards, which set down the 
minimum content of study programmes, student workload, 
and requirements students must meet to receive degrees. The 
state requirements are the core, to which regional 
requirements may be added. Based on these, higher education 
institutions develop their own study programmes. 

As described in the new Slovak Higher Education Act, 
higher education institutions can develop and implement their 
own programmes. The Ministry of Education administers the 
system of study fields in which higher education institutions in 
Slovakia may provide education. 

In Slovenia, the Higher Education Council of the Republic of 
Slovenia, founded in 1994, is responsible for defining the 
standards for the design of curricula. 

The 1996 Law on Education in Ukraine sets down the 
general standards of education. The Ministry is in the process 
of developing individual standards for each level of education 
and speciality. Higher education institutions are awarded the 
right by law to design their own curricula in line with the 
respective legislation. 

4.3.5. Quality Assurance 

It appears that in the Central and Eastern European countries, 
the predominant choice of “fitness for purpose” as the quality 
standard has emanated from the understanding that the 
purpose of a higher education institution or programme is 
defined in the pertaining legislation and that fitness is 
measured against the degree of compliance with the legislated 
requirements. That the mission statements of higher education 
institutions have been factored into quality evaluations from 
the beginning of the quality assessment process (e.g., in 
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Albania, Bulgaria, Estonia, Hungary, Latvia, Romania, 
Slovenia) was an indication of awareness that autonomous 
institutions must become more proactive in designing their 
futures rather than in checking up on how much they live up 
to their self-defined missions. 

In all the Central and Eastern European countries, quality 
assurance was introduced with the following purposes: 

- to protect stakeholders; 

- to define quality standards and levels; 

- to assure, to varying degrees, the comparability of study 
programme content and levels with those in Western 
Europe; and 

- in many countries, to control the quality of education at 
proliferating private institutions. 

The expansion in the number of private higher education 
institutions and consequently the concern for the quality of 
education provision were factors in introducing quality 
assessment, and specifically accreditation, at the system level 
in most Central and Eastern European countries (Albania, 
Bulgaria, the Czech Republic, Estonia, Latvia, Romania, 
Russia, and Ukraine). As an example, in the Czech Republic, 
twenty private institutions have been established since 1999; 
however, they only cater to 2.1 percent of the students in the 
countiy (Individual country reports: Czech Republic, 2002 
<http : / /WWW. ceenetwork. hu>) . 

In Hungary, the movement to establish private higher 
education institutions has not been as pronounced as in many 
other countries of Central and Eastern Europe. A few initiatives 
were thwarted by the Hungarian Accreditation Committee on 
the grounds that the quality of provision was not up to 
standard. Slovakia has only one private higher education 
institution. At the same time, however, the country faces a 
problem of maintaining the standard of quality of education 
because of a rise, over the last ten years, in the number of 
public universities. Slovenia, with orrly two universities, had 
two “free-standing higher education institutions” in 1995, and 
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seven, by 2001. However, the need for a methodical quality 
control mechanism is still under discussion. 

But while the proliferation of private institutions has given 
cause for concern with regard to quality education, the Polish 
University Accreditation Commission deals only with research 
universities, which private institutions are not. The quality 
control of private institutions in Poland will likely be among the 
responsibilities of the newly established National General 
Accreditation Commission. 

Steep increases in higher education enrollments have been 
registered in Poland and Hungary: a 3.5-fold increase between 
1990 and 2000 in Poland (Wojcicka and Chwirot, 2001) from 
10 percent to over 30 percent of the 18 to 23-year age cohort, 
and in the same period in Hungary (Statistical Yearbook of 
Hungary, 2000), from 1 12,000 students (Mokosin, 1999, p. 1 1) 
to 215,207 students (Individual Country Reports: Czech 
Republic, 2002 <http://www.ceenetwork.hu>). In Poland, 
however, there is an oversupply of study places so that higher 
education institutions must compete for new students. 

The universities that are considered to offer the highest 
quality education are only beginning to face this problem as 
the age cohort decreases. In the Czech Republic, “the total 
number of students graduating from Master’s degree study 
programmes has risen by about 62 percent since 1990” 
(Individual Country Reports: Czech Republic, 2002, <http: 
//www.ceenetwork.hu>). Even so, the number of applications, 
at least in the public universities, still exceeds the number of 
openings. 

In the other countries, the growth of student enrollments has 
had similar effects. It must be noted, however, that many expect 
the expansion to level off in the near future [e.g., Estonia) in 
response to demographic development. While most countries 
still have an oversupply of applicants for available study places 
at system level, some higher education institutions or study 
programmes are competing for students. Generally speaking, 
however, accreditation is not yet viewed as a publicly recognized 
factor for choosing a higher education institution. 
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It is evident that the preference in aU the Central and 
Eastern European countries for accreditation rather than 
quality assessment alone occurred because, at the time of 
transition, a priority was to establish some sort of quality 
control for the higher education sector. This priority was 
defined by higher education policy makers and stakeholders, 
both in government and in institutions. The question of how 
much external stakeholders acceded to this approach deserves 
more analysis. It is likely that at a time when, with the start of 
a new political system, new structures in higher education 
were developing at an accelerated pace, those “close to the 
fire”, te., the policy makers and those near them, had more 
sway than external individuals. 

Nevertheless, the quality assurance process as a whole is 
perceived as serving all stakeholders including those in the 
Central and Eastern European countries. The quality of higher 
education is of general concern. The methodology for assuring 
it continues to be debated and reconsidered. 

4.3.6. Quality Assurance Co-operation between Central and 
Eastern European Countries 

The Baltic States Co-operation for Quality Assurance in Higher 
Education was launched in 1994 within the Legislative Reform 
Programme for Higher Education of the Council of Europe. The 
project, which involved meetings between rectors and ministiy 
officials of Estonia, Latvia, and Lithuania, led to a co-operation 
agreement in the area of quality assessment between the 
ministers of education of the three countries. In accordance 
with the initial aim of the project, the three countries set up 
national quality assessment agencies in 1994 and 1995. The 
project has helped to clarify criteria, standards, and questions 
of methodology. In 1996, the Baltic Higher Education Co- 
ordination Committee adopted its statute. The nine members of 
the Committee are nominated by the national quality 
assessment body of each country, the respective Rectors’ 
Conference, and the respective Minister of Education (the 
nominee of the latter being the ENIC representative of the 
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countiy). The objectives of the Committee are to stimulate co- 
operation in institutional and programme assessment (tools 
and procedures for international reviewers), stimulate student 
and staff mobility, promote internationalization at higher 
education institutions, develop common higher education 
standards, publish assessment outcomes, and organize joint 
conferences and workshops. The Baltic co-operation has been 
successful in providing feedback into the quality assessment 
processes of the three countries and especially in the exchange 
of peer reviewers, which is ongoing. 

The Network of Central and Eastern European Quality 
Assurance Agencies in Higher Education (a Central and Eastern 
European Network) was established in 2000, at the same time 
as the Regional Sub-Network of the International Network of 
Quality Assurance Agencies in Higher Education. Although the 
link to INQAAHE was only an informal one, the members 
decided after a year to continue as an independent organization 
in name as weU. The intention of Network member agencies is to 
mutually share experiences and to foster co-operation, to 
exchange information, to recommend experts, to serve as a 
clearing house for quality assurance issues in Central and 
Eastern Europe, and to assist each other in elaborating 
measures for harmonizing quality assurance activities within 
the European higher education area. Member agencies are from 
Albania, Bulgaria, Croatia, the Czech Republic, Estonia, 
Hungary, Latvia, Lithuania, Macedonia, Poland, the Russian 
Federation, the Slovak Republic, and Slovenia. The application 
for membership of the Austrian Accreditation Council will be the 
object of a decision at the Assembly of the Central and Eastern 
European Network in October 2002. 

In 2001, the European Network for Quality Assurance 
(ENQA), an organization overseen by the European 
Commission, extended the possibility for membership to the 
associated countries of the European Union. Several Central 
and Eastern European Network member agencies were 
admitted to ENQA at its General Assembly in May 2002. As 
part of the application process for membership, a discrepancy 
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regarding t±ie concept of independence from government 
became evident between the European Union and the Central 
and Eastern European agencies. As described above, the 
Central and Eastern European agencies vary in this respect, 
from operating directly under ministries to acting as advisory 
bodies to the respective minister on accreditation decisions but 
making quality judgments autonomously. It appears that the 
definitions behind the common terminology require clarification, 
without disregard for the social and historical context. 

4.3.7. Expected Trends 

It is very difficult to say in which direction the legislative 
process regarding higher education is going. It seems that in 
this sector, too, the initial feeling that “everything is possible”, 
which pervaded public sentiment in the early days of 
democratic government in Central and Eastern Europe, was 
not able to shed the frame of mind ingrained both by decades 
of control and a more authoritative tradition relative to^ say, 
northern Europe in general. Often, more autonomy in higher 
education is at once coupled with detailed regulation, which, 
with the inevitable amendments to the new laws, is becoming 
more intricate rather than more general. To steer higher 
education institutions towards “defining their own mission and 
establishing direct links to society and its changing needs” 
(Tomusk, 2000, p. 182); moreover, to give them the financial 
independence that this objective requires is a prerequisite for 
the organic development of the sector. 

An outlook on higher education developments and current 
trends in each CEE country goes beyond the scope of this 
example. Hungary may serve as a case study. Here, a financial 
reform project for higher education has been commissioned by 
the government, in which the weaknesses of the overly 
regulated system have been defined (Matolcsy, 2001, pp. 19- 
20). These weaknesses include low cost effectiveness and 
efficiency; no provision for an external board to oversee an 
institution while the main governing body, the academic 
senate, has a stake in upholding the status quo; educational 
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provision decided by tradition and academic staff preferences; 
very limited own income and consequently little impact on 
society on the part of institutions; personal “entrepreneurship” 
on the part of the academic staff using institutional facilities, 
such as running after-hours courses or teaching at several 
places with full pay from each; a frequently changing legislative 
environment making long-term planning difficult; and 
dependence on the national budget law which earmarks 
spending and precludes innovative financial strategies 
(Matolcsy, 2002, p. 22). 

Another development in the arena of higher education in 
Hungary is a debate conducted in the commentary section of 
the weekly Elet es Irodalom [Life and Literature] (18 January 
2002 et seq.), launched with the publication of a book by two 
Hungarian academics (Polonyl and Timar, 200 1) on the state of 
higher education in the country. They cite an OECD (SIALS) 
survey on reading comprehension in twenty-two countries. 
While Hungarians generally are proud of the highly educated 
individuals they have given to the world of science, the study 
puts Hungarian 14-to-64-year-olds in last place. The analysis 
concludes that, on the one hand, a narrow elite is and was 
indeed well educated, but that, on the other hand, education 
in Hungary is geared toward cramming factual information 
rather than toward problem-solving and innovative thinking. 

A similar trend, incidentally, may be observed elsewhere in 
the region (Tomusk, 2000, p. 183). Half a dozen education 
researchers, sociologists, economists, and others have joined 
the debate in the paper to reflect not only on the - commonly 
knovm - miserable state of the higher and public education 
infrastructure and the consequences of under-funding the 
sector overall, but on the structural distortions that have 
hardly been addressed up until now and certainly not as 
broadly as in the current discussion. 

With regard to the higher education sector, it appears that 
there is a natural curve of development, from launching a new 
legislative system to “improving” it until it becomes saturated, 
to recognizing the failure of the model, to identifying its 



84 C. CAMPBELL and C. ROZSNYAI 



weaknesses, and finally, to restructuring the system. Hungaiy 
can be said to be approaching the peak of the curve. 
Legislation is still being “refined”, but the core weaknesses of 
the model are already being identified. 

With regard to programme design, a problem with the 
introduction of the two-level study system is that the 
employment market in the CEE countries is not ready to 
absorb graduates of the first level. This situation may, 
however, be a consequence of the fact that the outcome in 
content of the first-level degree has not been defined. So-called 
“Bachelor’s” Degree holders, ie., college graduates in dual 
systems, are not faced with this problem. Therefore it can be 
expected that as curricula are revised to accommodate the 
two-level structure of studies and bachelor-level graduates of 
this type enter the emplo 3 rment market, it will be ready. 

It is generally the case for the higher education sector in 
Central and Eastern European countries that the greatest 
problem continues to be the lack of adequate financial 
resources. While that problem is the object of prevalent 
complaints in the West, the funding in real terms is far higher 
there than in Central and Eastern Europe. Moreover, the 
condition of the infrastructure and the cost-effectiveness of the 
sector overall lag far behind. In Central and Eastern Europe, 
the average salary of academic staff members is a fraction of 
that of their Western colleagues, which has far-reaching 
consequences on many aspects of higher education. It should 
be remembered, however, that enormous progress has been 
made in the last decade concerning structural developments in 
higher education, as has been described, accompanied by a 
steady rise in the GDP in many Central and Eastern European 
countries. Together, the two trends may provide the sector in 
the region with opportunities to define its future, both in the 
countries individually and as part of the European higher 
education area. 
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Annex 1 

Resources Chart from 
the December 2001 INgAAHE Sm^ey 



This chart summarizes published responses to two of the 
questions in the December 2001 INQAAHE survey of its 
members asking: 

- At whose initiative the external evaluation agency was 
established? 

- What, in the view of the Agency, was the relative 
importance of the purposes of their evaluations? (The 
replies were rated on a scale of 1-5, with 5 being most 
important.) 

- It indicates which providers of education are covered by 
the evaluation processes of the Agency. 

- It only lists respondents (from European Union member 
states) who are full members of INQAAHE. 

- They are also members of ENQA. 



o 





4 



87 



88 C. CAMPBELL and C. ROZSNYAI 



Agency: name and 


Established Constituency 


Accoimtability Accreditation 


Enhancement 


Benclimarking Info 


country /region 


at 














Austria 

- Osterreichischer 


Government 


Private universities 


5 




3 


4 


2 


Akkreditieruj-igsmt 


initiative 














Austria 

- Gesdieliajfsselle 


Government 


Fachhodisdmlen 


5 




4 






Fachhochschulmt 


initiative 














Belgium 
- Vlaamse 


Government 


Hogescholen 












Hogescholenrad 


initiative 














Denmark 

-Danmarks 


Government 


All levels of education 


3 




5 


5 


5 


Eualueringsinstitut 

Finland 


initiative 














- Finnish Higher 
Education 




Polytechnics and 


3 




5 


3 




Evaluation Council 


Government 


universities 












FINHEEC 


initiative 














France 

- (^mile National 


Government 


All higlier education 






5 






dEixilualion 


initiative 


institutions 












Germany 


Joint 














- Akkrediterungsrat Government 














National 


and 

institutional 

initiative 


Accreditation agencies 




5 


4 




3 


Norway 

- Network Norway 


Government 




3 




5 




4 


Council 


initiative 














Portugal 


Government 


Initial teacher 






Q 






’INAFOP 


initiative 


education programmes 




0 


O 






Netherlands 
> HBO-Raad 


Hogscholen 

initiative 


Hogesdiolen 


5 


Not until 
2002-2003 


5 


3 


4 


Netherlands 

>VSNU 


University 

initiative 


Universities 


3 


Not until 
2002-2003 


5 


4 


1 


Sweden 




Universities/ 












- National Agency 




university colleges and 












for Higher 


Government 


professional 






0 






Education 


initiative 


accreditation 












United Kingdom 


Joint 














’ Quality Assurance government/ 
Agency for Higher funding 
Education council and 


Universities, colleges of 
higher education, and 














higher 

education 


higher education 
furiJier education 


5 




5 




5 




institutions 

initiative 


colleges 













o 

ERIC 

I 



85 



best copy available 



Annex 2 



Descriptions of Higher Education Institutions 
and Quality Assurance in Central and Eastern 

European Countries 



2.1. ALBANIA 

Albania has eight universities, three of which are in Tirana, 
and five, in other parts of the country. There are also two 
academies, for art and for sports, and one “higher school” for 
nursing. In addition, there is a military and a police academy. 
There are no private higher education institutions, even 
though the legislation makes their establishment possible. A 
separate law on private institutions is foreseen. All institutions 
must have state recognition to operate. The number of 
applying students exceeds the number of study places overall; 
however, as everywhere in the world, there is little interest in 
certain subjects and an overabundance of applications for 
such popular subjects as economics and law. The government 
determines the number of study places as proposed by the 
institutions. Tuition fees have recently been introduced, of 
which the institutions can retain up to 90 percent with the 
balance going to the central budget. These fees are 
accompanied by a grant system that is based on family income 
and merit. 

The Accreditation Agency of Higher Education, along with 
the Council of Accreditation, was formally established in 1999, 
with the passing of the higher education law and specifically a 
decision by the Council of Ministers. However, Albania is just 
launching the accreditation process with the assistance of a 
just-ending, three-year TEMPUS project. The aim is for the 
same evaluation team to assess similar programmes in the 
whole country. At programme level, the desire is to assess both 
educational and research activities. The scope of assessments 
is very broad, encompassing both institutional and programme 
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levels, and within these, an extensive number of factors, from 
goals and aims to the opinion of society at large. One can 
expect that once the agency has gained some experience the 
scope will become more focused. The Agency aims to evaluate 
institutions and programmes every four years. 

The Accreditation Council has eleven members. They are 
distinguished academics, including two representatives from 
industry and one from the Ministry of Education and Science, 
who are named by the Council of Ministers on the proposal of 
the Minister of Education and Science. The Accreditation 
Council approves the criteria of self-evaluation and external 
evaluation and decides on the accreditation report. The 
Accreditation Agency has a staff of five, one director, three 
programme officers, and one administrator. The programme 
officers participate as secretaries in the review visit and 
contribute to writing the accreditation report. The report, 
which describes the strengths and weaknesses of the 
evaluated institution, is submitted to the Council of 
Accreditation, and the decision of the latter is presented to the 
Minister, who has the final decision on granting accreditation. 
The report is published but distributed only within the sector. 

The Agency prepares the criteria for evaluation to be 
approved by the Council. The Agency issues two manuals, one 
for self-evaluation, used by the institutions, and the other for 
the review team. It is interesting to note that the Agency trains 
the self-evaluation team at the institutions prior to launching 
the self-evaluation exercise. Equally interesting is the provision 
in the law that the self-evaluation should create the basis for a 
development plan for the institution. The Agency also trains 
the external reviewers who, as stated in the law, come from the 
organizations of employers, industry, and other universities. 

2.2. BULGARIA 

Bulgaria has forty-nine higher education institutions that 
include thirty-seven state universities and specialized higher 
schools, four private universities, two state, and six private 
colleges. All higher education institutions operating in Bulgaria 



er|c 



87 



QUALFIY ASSURANCE 91 



must be recognized by the National Assembly. The training for 
Doctoral Degrees provided in connection with research 
specialities may also be offered by the Bulgarian Academy of 
Sciences, the Agricultural Academy, and the other research 
organizations in the country. 

The National Evaluation and Accreditation Agency conducts 
institutional and programme accreditation. The law stipulates 
that programme accreditation can only be conducted in 
institutions that have already been accredited. Programme 
accreditation is conducted for programmes leading to 
educational and professional degrees (“Specialist in...”), 
educational qualification (Bachelor’s and Master’s Degrees), as 
well as for education and research (Doctoral Degrees). The 
Agency also accredits new programmes to be launched in 
accredited institutions. Doctoral education may be conducted 
at universities and research organizations which have received 
the highest of four ratings in their accreditation of research 
specialities for Doctoral Degrees. 

The Agency evaluates projects for the opening and 
transforming of a higher school, a faculty, an institute, a 
branch, or a college as well as for the opening of new 
specialities. 

The Accreditation Council consists of nine recognized 
academics, including two representing the Minister of 
Education and Science. The Council Chairperson is also the 
Chairperson of the Agency. The Council is appointed for a six- 
year term. The Council sets up standing committees of three to 
seven members appointed by the Chairperson for a three-year 
term. The role of the standing committees is to review the 
reports of the expert groups that have conducted site visits. 
The Standing Committee evaluates the report on the strengths 
and weaknesses of the institution or programme described and 
proposes an accreditation grade. 

The Agency has a full-time staff of twenty persons Who have 
an administrative and coordinating role. They do not 
participate in the review visits, but do contribute to the 
accreditation report. The Agency develops its evaluation 
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criteria and procedures in accordance with the law and other 
legislation, both for institutional and programme reviews. It 
also organizes workshops for training its review teams. The 
results of accreditation are published annually in the State 
Gazette, while the reports are published periodically in the 
Bulletin of the Agency and are distributed to higher education 
institutions, research organizations, ministries, and other state 
organizations. 

2.3. CROATIA 

Croatia has four multi-faculty universities, which also include 
academies and polytechnic schools. In 2001, the Ministry of 
Science and Technology decreed that polytechnics might also 
be established as independent institutions. Consequently 
seven polytechnics and eight independent schools have been 
established. 

The national system of quality assurance is set down in the 
1993 Higher Education Act. It was amended in 1996 to 
separate the functions of assessment and accreditation. A new 
draft law is currently under discussion. 

The Ministry initiates the accreditation of study 
programmes and makes accreditation decisions, based on the 
recommendations of the National Council for Higher 
Education. The Council has eighteen members and a 
President. On nomination by the Rectors Conference and 
higher education institutions, the members and the President 
of the Council are appointed by the Parliament of the Republic 
of Croatia for a four-year term. The role of the Council is to 
propose the experts, and following their assessment, to 
formulate a recommendation to the Ministry regarding 
accreditation. The same review team conducts the review of all 
programmes in the same general subject area in the country. 

Higher education institutions are evaluated without a 
“formal” accreditation decision. Nevertheless, the Council may 
propose the closing of an institution that does not meet quality 
standards. The Council devises guidelines for self-evaluation 
and the framework for review, but the Ministry approves the 
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procedures. The Ministiy has the function of a quality agency 
in that it provides the logistics for reviews. The review team is 
appointed by the Ministry on the proposal of the Council. It 
consists of academics, some of them (mostly of Croatian origin) 
are from abroad. It submits its evaluation report to the 
ministry, which forwards it to the Council. The Council then 
debates the content of the team’s report, based on the self- 
evaluation report, the opinions of professional organizations 
and international experts, and may accept it as is or require 
additional information. The Council then formulates one of 
four proposals: from ruling that the institution meets the 
quality standards, to requesting the implementation of 
changes by a given date, to changing the status (funding) of 
the institution, or to closing it. The Ministiy makes the final 
decision. 

The tasks of the Council, in addition to quality assessment, 
also extend to commenting on higher education policy and 
strategy and proposing to initiate new or to abolish existing 
course programmes and higher education institutions. 

2.4. THE CZECH REPUBLIC 

The Accreditation Commission of the Czech Republic was 
established by means of the Higher Education Act in 1990. Its 
twenty-one members, including two from business and three 
from foreign countries, are appointed by the Government, on 
the nomination of the Minister of Education, Youth, and 
Sports, from higher education institutions, the Academy of 
Sciences, and the Research and Development Council of the 
Government. The Minister must consult with representative 
bodies of the three groups. Commission members are 
appointed for four- or six-year renewable terms. Additionally, 
chairpersons of permanent working groups cover disciplines 
not represented by commission members. Ministiy 
representatives may attend Commission meetings. A person 
from the appl^dng institution or programme may be invited 
when the Commission discusses the application. 
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In 1999, a Secretariat was set up and has since expanded 
to include a staff of six, including the Secretary. It is 
technically a unit of the Department for Science and Higher 
Education of the Ministry of Education, Youth, and Sport. The 
Commission Secretary is appointed by the Minister on the 
proposal of the Chairperson of the Commission. The staff 
performs a coordinating and administrative function, while the 
Commission members, who meet to discuss and vote on 
applications and higher education issues “at least three times 
a year” (The Statute of the Accreditation Committee, Ministry 
of Education Youth and Sports of the Czech Republic. Article 2 
[1]. p. 1) but in fact more often, carry the full load of work in 
preparing the evaluations. 

At the end of 2001, the Czech Republic had forty-eight 
higher education institutions of which twenty-four were public, 
four were state institutions (one police academy and three 
military institutions), and twenty were private institutions. The 
public and military institutions are universities with a research 
orientation and offer Bachelor’s, Master’s, and Doctoral 
programmes. The Police Academy and all the private 
institutions are non-university vocational institutions offering 
Bachelor-level degrees. 

In 1998, the Higher Education Act was amended so as to 
greatly expand the mandate of the Commission. While earlier, 
only new programmes were accredited without provision for 
ongoing quality control, evaluation cycles were now 
introduced. The scope of the commission is to evaluate 
“applications for the accreditation of study programmes and... 
for the accreditation of procedures for obtaining venium docenti 
(habilitation) or procedures for appointment of professors in a 
given field, ...[the] establishment, merger, amalgamation, 
splitting, or dissolution of a faculty of a public or state higher 
education institution, granting the state permission for a legal 
entity desiring to operate as a private higher education 
institution, determining the type of higher education 
institution” (The Statute of the Accreditation Commission, 
Ministry of Education Youth and Sports of the Czech Republic. 
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Article 2 [3] and [4b-d], pp. 1-2). The Accreditation 

Commission also has an indirect influence on higher education 
policy in that it is asked to “assess other issues concerning the 
system of higher education presented to it by the Minister and 
express its standpoints [regarding] these issues” fThe Statute 
of the Accreditation Commission, Ministry of Education Youth 
and Sports of the Czech Republic, Article 1 [1] b, p. 1). 

The commission and its working groups devise their own 
standards for evaluation and accreditation, based on the 
Higher Education Law and pertinent legislation. 

While the denial of accreditation can lead to the closing of 
an institution or programme, the commission invokes this 
right only in cases in which no hope for improvement is 
perceived. It may also propose to restrict accreditation so that 
no new students are admitted, or withdraw accreditation so 
that no degree may be awarded until the programme is 
improved. 

Reports include a description of strengths and weaknesses 
and contain a number of specific provisions for improving the 
quality of a programme or institution. Higher education 
institutions are required by law to have internal quality control 
mechanisms and to publish the outcomes of their reviews once 
a year (a provision in the 1998 amendment to the Law). In its 
external evaluation, the Accreditation Commission considers 
the capacity of the given institution to control its quality. 

2.5. ESTONIA 

Estonia has six public universities, seven state applied higher 
education institutions (polytechnics), six state higher 
vocational institutions, eight private higher vocational 
institutions, and twenty- two private higher education 
institutions. They are required to obtain a license from the 
Ministry of Culture and Education to operate and may issue 
state-recognized degrees only for study programmes that have 
been accredited. 

The Higher Education Accreditation Center was established 
by the Government in 1997 as a foundation. Its primary 
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activity is to organize the accreditation of course programmes; 
however, it has also reviewed some institutions. The Center 
organizes research evaluation separately. 

The Center functions as the coordinator for accreditation 
procedures with a staff of three: a Head and two Assistant 
Heads, one for research and the other for evaluation. The role 
of the Center is to identify experts and to organize their review 
visits. The review teams are made up of only foreign experts. 
The review team schedules are organized so that the team is 
able to review all the programmes in the broad given field in 
the country within the one week allotted to their visit, which 
includes writing the review report. 

The Higher Education Quality Assessment Council of 
Estonia was established by the 1995 Law on Universities, to 
“operate under the jurisdiction of the Ministry of Culture and 
Education”. It has twelve members, including members from 
other Nordic countries, who are “representatives from amongst 
research and development institutions, state foundations that 
finance research and development activity, and from 
professional associations”. They are appointed by the 
Government for a three-year term that may be renewed once. 
The Council votes on the review team reports and is 
responsible for devising the criteria and methodology for 
accreditation which are approved by the Minister. Government 
regulations set down the general standards for curricula and 
levels of study. The accreditation decision is either for full 
accreditation (valid for seven years), conditional accreditation 
(valid for two years), or not accredited. If over one-third of the 
prograimmes of a university are not accredited, its license is 
revoked. (For research evaluation the possible grades that are 
awarded are excellent, good, satisfactory, or unsatisfactory). 
The judgments of the Council are advisory to the Minister of 
Education, who makes the final decision regarding 
accreditation of a programme or an institution. The Minister 
has veto power over Council decisions but may not change 
them and is required to explain his or her reasons. Reports 
deal with the strengths and weaknesses of the evaluated 
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programme or institution. They are not widely diffused, but 
printed copies are distributed to universities and student 
organizations. 

2.6. HUNGARY 

The Hungarian Accreditation Committee was established as 
the result of the first Hungarian Higher Education Law in 
1993. Its scope is to accredit higher education institutions, 
which, in the first round, that ended in 2000, it did by 
assessing all study programmes at given evaluated 
institutions. In the coming round of evaluations, the focus will 
be more on the institution as a whole, and only a selection of 
programmes will be evaluated. The evaluation of entire fields at 
one time and by expert reviewers from a consistent pool is also 
being considered, the purpose being the comparability of study 
programmes in the country. The Hungarian Accreditation 
Committee adopted a strategic plan in early 2002 and is 
proposing a new procedure in consultation with higher 
education institutions. The Committee conducts programme 
evaluation according to the standards devised by its expert 
committees for each discipline. 

In addition, the Hungarian Accreditation Committee 
evaluates applications to establish and grant state recognition 
to higher education institutions and faculties, on new study 
programmes, and on national qualification requirements for all 
degree programmes taught in Hungary. The Committee also 
advises on habilitation regulations. According to the 
amendment of 2000 to the Higher Education Act, the 
Committee is also charged with evaluating applications for 
professorships. 

Hungary has eighteen state universities, twelve colleges, five 
church-maintained universities, twenty-one church- 
funded /managed colleges, and six private colleges. Hungary 
has a dual tertiary system with universities providing 
research-based and research-oriented education and with 
colleges offering applied training. Student movement from one 
system to the other, although provided for by law, has been 
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difficult and inirequent, except in the few cases in which 
bilateral agreements between certain institutions facilitate it. 
The sector is discussing a number of options to introduce the 
Bachelor’s/Master’s model. The introduction of an ECTS- 
compatible credit system is underway. 

In order to receive state recognition and to award degrees, 
private higher education institutions must undergo accreditation. 
Church-maintained higher education institutions receive state 
financing similar to state institutions and are accredited, but 
the mandate of the Committee only pertains to secular 
programmes. 

The decisions of the Committee regarding doctoral schools 
is binding. All its other decisions are advisory, but in case the 
Minister of Eklucation deviates from the assessment of the 
Committee, he or she must publish the reasons for his or her 
decision. The scope of the Hungarian Accreditation Committee 
is accreditation oriented toward improvement. Accreditation for 
higher education institutions is contingent on compliance with 
the Higher Education Act. In the first round of institutional 
accreditation, the evaluated programmes were graded on a 
four-level scale. Roughly one third of the programmes were 
awarded accreditation for a specified period with prescribed 
requirements to be met by that date. These requirements were 
reviewed in a monitoring procedure. In the next institutional 
accreditation round, programmes will be given one of three 
grades: “yes” (may be accredited), “conditional yes” (may be 
accredited but is required to meet stated requirements by a 
stated date), and “no” (may not be accredited). 

In the first round, only the final accreditation decision was 
published along with a brief explanation. In the following 
round, the full report will be published. 

The Hungarian Accreditation Committee consists of thirty 
full members delegated by higher education institutions, 
research institutes, and professional organizations. They 
receive their letters of appointment from the Prime Minister for 
a three-year term, which may be renewed once. There is one 
non-voting student member. Additional non-voting members 
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are appointed in order to cover all the main disciplines. An 
International Advisory Board, mandated by law, reviews the 
work of the Committee at an annual meeting and makes 
recommendations which are also sent to the Minister. 

The Secretariat has a staff of sixteen programme officers, 
four of whom are part-time, and nine adrninistrative staff. The 
programme officers are assigned several expert committees the 
work of which they prepare and guide from inception to 
completion of the report. In addition, they are responsible for 
several institutions; they prepare and participate in site-visits; 
and they assist in writing the accreditation report. 

In 2000-2001, the Hungarian Accreditation Committee was 
evaluated by an international review team under the auspices 
of the European Rectors Conference (CRE, now the European 
University Association EUA). Its report, together with the reply 
of the Committee, have been published (<http://www.mab.hu/ 
english/a_links.html>) . 

2.7. LATVIA 

There are forty higher education institutions in Latvia, of 
which twenty-three are state institutions and seventeen are 
private. All five universities are state institutions. 

The Higher Education Quality Evaluation Center, Ltd., is 
owned by shareholders. These are the Ministry of Education 
and Science and five higher education institutions. A five- 
member board supervises its activities. One member of this 
board is appointed by the Ministiy. The shareholders elect the 
Director of the Center who hires the staff. The staff includes 
two academics and consists of four full-time equivalent 
positions, including the Director, which are filled by two full- 
time and four part-time employees. 

The Center has a primarily coordinating function. It sets the 
timetable for reviews in consultation with higher education 
institutions and brings together the evaluation teams as 
required by the Council of Higher Education (for institutions) 
or the Accreditation Commission (for programme assessment), 
in consultation with the institutions or persons responsible for 
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programmes to be reviewed. The staff of the Center does not 
participate in the visits. While there is no national-level 
assessment of all study programmes in a field, the Center does 
attempt to use the same experts to evaluate similar 
programmes. The Center has guidelines for self-evaluation and 
external assessment, but the evaluation team has great 
autonomy in deciding on its own methodology and procedures, 
so long as they comply with the legislated standards. The 
teams are made up only of academics, one from Latvia and at 
least two foreign team members. Only for college -level 
professional education may all the experts be from Latvia. The 
review team reports are published without changes, and the 
teams aire fully responsible for their content. The evaluation 
report is sent directly to the higher education institution as 
well as to the Council for Higher Education and the 
Accreditation Commission, and is published. 

The Council for Higher Education is set up by Parliament, 
and the Accreditation Commission, by the Ministry. The two 
bodies approve the review teams set up by the Center and 
decide whether accreditation should be granted. The Council 
for Higher Education Institutions and the Commission for 
Study Programmes approve the reports of the independent 
evaluation teams organized by the Center. They submit their 
decisions to the Minister, who may refuse to grant 
accreditation simply on the grounds of a legal violation. 

The Council for Higher education is an independent 
institution. It has ten members, including its full-time 
Chairperson who is prohibited from holding a leadership 
position elsewhere. They are delegated by bodies representing 
higher education, research, professional organizations, and 
industry. The Minister of Education and Science also attends 
the meetings. The members are approved by Parliament for 
four-year terms. In addition to adopting decisioris on the 
accreditation of higher education institutions, the Council 
undertakes a wide range of duties regarding higher education 
strategy, quality, and financing. 



QUAinr ASSURANCE 101 



The Accreditation Commission is set up by the Minister to 
assess study programmes and to propose an accreditation 
decision to him or her. Its members are delegated by higher 
education institutions, research institutes, and the Ministry, 
as well as by unions, professional associations, and other 
ministries. 

Institutions are granted permanent accreditation status 
except in cases in which improvement is needed. In such 
cases, accreditation is given for two years. New study 
programmes must receive accreditation within two years of 
starting activities. Study programmes are accredited for six 
years, but may receive a two-year accreditation once if they 
need to be improved. The Council or Commission may also 
decide that accreditation should be refused. After repeated 
negative accreditation, the institution or study programme 
must be reorganized or closed, depending on the severity of the 
problems. Negative as well as positive decisions are published. 

New study programmes must be granted licenses before 
they can operate and must receive accreditation within two 
years. Study programmes must be accredited every six years, 
at the least. 

2.8. LITHUANIA 

During the 2001-2002 academic year, there were twenty-two 
state and thirteen non-state higher education institutions 
operating in Lithuania. They include nineteen university-type 
institutions (fifteen state and four non-state) and sixteen non- 
university-type institutions — colleges (seven — state and six 
non-state). 

The Lithuanian Center for Quality Assessment in Higher 
Education was established, and is financed, by government, 
but the agency functions independently. The Minister for 
Science and Education appoints the Director of the Center on 
a competitive basis to serve a five-year term. 

The Agency staff organizes institutional and programme 
assessment. In addition, it conducts research evaluation 
separately. The activities of the Center focus on quality 
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assessment and improvement, even though the Center 
forwards its assessment reports to the Ministry. 

The Ministry sets up a Council of Experts. The latter is 
responsible for summarizing the conclusions of the 
assessments. The external evaluation schedule for study 
programmes, institutions, and the subdivision is deterrtiined 
by the Center in consultation with the Rectors’ Conference, the 
College Directors’ Conference, the Directors’ Conference of 
State Research Institutes, and the Science Council. The 
assessment plans are approved by the Minister. Institutional 
and programme accreditation decisions are taken by the 
Mimster of Science and Education, based on the report of the 
review team, approved by the Council of Experts. Accreditation 
is mandatory. 

The agency functions as the coordinator of reviews. It also 
organizes seminars for self-assessment. The regular 
assessment of education programmes began in 1999. The 
Agency appoints the review teams in consultation with the 
institution to be assessed. The review teams, which consist of 
academics and non-academics, prepare their reports for the 
evaluated institution. For the subject areas (especially the 
social sciences), international expert groups are formed. One or 
two local experts are included in each group. The review teams 
may or may not be the same for similar programmes. However, 
the same review team is used to evaluate all study 
programmes in the same general field in all higher education 
institutions. Programmes, which receive temporary or 
restricted assessment, are re-evaluated within two years. The 
teams hold responsibility for the report, and only a summary is 
published. The Center forwards the report to the evaluated 
institution and to the Council. 

While assessing the research and development of research 
and higher education institutions (or their subdivisions), 
efficiency of research activity may be assessed considering not 
only quantity indicators, but also quality parameters for 
research efficiency. The most important research results are 
determined following a debate on their significance. After every 
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assessment, the Agency publishes the research and 
development efficiency indicators of institutions of research 
and higher education taken from their self-assessment reports 
and supplemented with corrections according to the results of 
external assessment. 

The qualitative assessment of the whole institution of 
research and higher education embraces all its academic 
subdivisions as well as the whole combination of all its 
academic (educational and research) activities. Both an 
internal and external assessment of the whole institution are 
based on the previous assessment reports concerning the 
educational programmes as well as those of research in 
different subject areas, with the reports being summarized, 
and strategic matters, considered. Only in the case of a small 
institution may a general qualitative assessment of research 
and higher education coincide with an assessment of its 
research or study programmes. Summarized quality 
assessments include grading on a four-point scale. 

It is interesting to note that the Agency shares its offices, 
administration. Director, and Deputy-Director with the 
National Recognition of Qualifications and Information Center, 
the ENIC/NARIC, which allows the organizations to share their 
pools of experts. The Agency works with a staff of eight, but 
shares its administration with the Recognition Center. The 
quality assessment unit has a staff of four, including its Head, 
an academic holding a part-time position. 

2.9. MACEDONIA 

Macedonia recognizes four types of higher education 
institutions: universities, faculties, art academies, and higher 
vocational schools. Currently, the country has two state- 
recognized universities. There is also one private, recognized, 
foreign university. However, altogether, there are thirty-five 
higher education institutions. All must have licenses to 
operate. For the most attractive programmes, there are more 
applicants than available study places. 
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Currently, a project with international co-operation, the 
involvement of the evaluation agency, the Ministry of 
Education and Science, and the two state universities, is 
underway to design the quality assessment landscape of 
higher education in the counhy. The main focus is on 
introducing self-evaluation into higher education institutions. 

After ten years of preparation, the Law on Higher Education 
was adopted in August 2000. It called for the international 
comparability of higher education and introduced quality 
assessment and accreditation. Both institutions and 
prograimmes are to be evaluated and accredited. 

The Law set up two bodies for quality assessment, the 
Accreditation Board (also translated as “Licensing Board”) and 
the Evaluation Agency. The Board is an independent body of 
fifteen academics delegated by the Inter-University Conference, 
the Academy of Science and Art, and the Government for a 
four-year term, renewable once. It is responsible for 
determining whether or not a higher education institution is 
compl)dng with the general requirements set for it. It decides 
on the accreditation of new study programmes, keeps records 
on licensed higher education institutions, and assesses the 
quality of higher education in general. The Board may 
establish expert committees. 

The Evaluation Agency has nine members who are 
academics proposed by the Inter-University Conference and 
elected for a four-year, once renewable term by the Board by 
secret ballot. They represent proportionally the number of 
faculties and scientific institutions involved in higher 
education at all levels. The Agency functions as an 
independent and separate body but is convened by the 
President of the Board. It is responsible for conducting external 
evaluations based on guidelines adopted on the proposal of the 
Inter-University Conference. It should also review the quality of 
the academic staff in the higher education institutions of the 
country. It must monitor the quality of work at higher education 
institutions. Once every five years, it assesses the quality of 
higher education institutions; it gives recommendations for 
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improvement; it assesses the quality of the academic staff 
throughout the country and makes suggestions for 
improvement; it publishes its reports on assessment results 
which it transmits to the institutions, the Board, the Ministry, 
and the Parliament. Based on its findings, it recommends the 
extension or the withdrawal of the accreditation of given 
institutions. 

The law requires higher education institutions to conduct 
self-evaluations via committees set up for this purpose. 
General Guidelines for Evaluation and Evaluation Procedures 
at Universities have been published, and the Accreditation 
Board as well as the Evaluation Agency have issued their 
respective by-laws. 

2.10. POLAND 

In Poland, the proliferation of private higher education 
institutions has been enormous. Over 200 such institutions 
were established over the last decade (Wojcicka and Chwirot, 
2001, p.7). This result, in part, contributed to the 

establishment of a number of accreditation bodies. Business 
Schools for Quality of Higher Education (FORUM) was set up 
in 1994. The Accreditation Commission of Higher Vocational 
Education was established by the Law on Higher Professional 
Schools in 1997 and by a ministerial decree of 1998. The 
Accreditation Commission for Medical Universities was set up 
in 1997; the University Accreditation Commission, in 1998; the 
Accreditation Commission for Schools of Physical Education 
and one for Pedagogical Schools, in 1999. The Accreditation 
Commission of Technical Universities, one for Agricultural 
Universities, and the Accreditation Commission of the 
Foundation for the Promotion and Accreditation of Studies in 
Economics were set up in 2001. The aim of these commissions 
is to control the standard of study programmes in their given 
fields. 

The General Council on Higher Education was established 
as a result of the Polish Higher Education Law of 1990. One of 
its functions was to set the standards for new higher education 
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institutions, faculties, and study programmes. But “deprived 
of... organizational support and adequate financial means [it] 
has not been able to conduct systematic quality evaluation of 
provision in the hitherto existing and newly established 
institutions of tertiary education” (Wojcicka and Chwirot, 
2001, p. 7), In the 2001 amendment to the Higher Education 
Law, the likely successor to the General Council, the National 
General Accreditation Commission, was established. It began 
to function in January 2002 and is expected to work out 
threshold standards for quality. “In this situation, it seems that 
one of the more important tasks of the State Accreditation 
Commission [now translated as “National General Accreditation 
Commission”) is, in addition to seeing to the authorization of 
activities undertaken by the academic milieu via its official 
certification and limited co-ordination role, also assuring the 
provision of certain minimal quality standards by each tertiary 
education establishment. Such activities will allow for (i) 
developing important social initiatives in the academic milieu 
possessing self-regulatory character, (li) distinguishing and 
promoting institutions granting excellent provision, and (Hi) 
accelerating the inclusion into the evaluation system of all 
higher education institutions in Poland” (Wojcicka and 
Chwirot, 2001, p. 8). 

Following years of discussion about setting up a quality 
assurance system, the Conference of Rectors of Polish 
Universities (CRPU) took the initiative and established the 
University Accreditation Commission (UAC) in 1998. Its 
mission is the “enactment of quality standards in education 
comparable to those prevailing in the European Union; the 
upgrading of the quality of education; [and the] promotion of 
high-quality coui^ses of studies and of schools offering them” (2001, 
<http://www.main.amu.edu.pl/-ects/uka/uka-eng.html>). The 
membership is made up of seventeen “Vice-Rectors or 
representatives of the schools-signatories of the Agreement 
appointed by the Rectors of their schools and one person 
designated by CRPU. CRPU appoints the President of the 
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UAC. The tenure of UAC members is three years” 
(<http:/ /www.maln. amu.edu. pi/ -ects/uka/uka-eng.html>). 

The UAC specifically aims to set up a system of 
accreditation for degree programmes and to establish common 
standards of quality. Other institutions that are not members 
of the agreement may apply for accreditation of their 
programmes by the UAC but the UAC deals only with research 
universities. Some private institutions have, in fact, undergone 
accreditation by UAC. In their evaluation, for which the 
application is voluntary, the UAC looks at the curriculum, the 
academic qualifications of the teaching staff, the student/ 
teacher ratio, the adequacy of infrastructure, including library 
holdings and laboratories, as well as research or scholarly 
contacts, and the existence of student evaluations of the 
taught courses. 

At least five universities must apply for the accreditation of 
a particular study field in order to launch the accreditation 
procedure. The university applying for accreditation proposes 
five experts with documented qualifications to participate in 
the evaluation (two from the university and three from another 
university). An expert group of six to twelve persons, whose 
task is to establish specific standards for evaluating the study 
field, and a three- to five-member evaluation team for each 
applying programme, that reviews the programme and writes 
an evaluation report, is set up. The evaluation team may 
recommend that the UAC grant accreditation (for two to five 
years), prescribe certain requirements before accreditation may 
be granted, or refuse accreditation. The expert group reviews 
and updates its standards at least every two years. The UAC 
employs two full-time and one part-time staff members. These 
perform coordinating and administrative functions. 

The proliferation of accreditation commissions has 
galvanized their members into establishing an umbrella 
organization. It is currently developing. Its members are the 
chairpeople of the seven academic accreditation commissions. 
Its tasks are to co-ordinate the activities of its seven member 
commissions, particularly to observe that accreditation 
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standards and procedures are correct and comparable, and 
that fields of study offered by different schools are accredited 
under the same standards and procedures. It should also 
represent the seven commissions vis-d-vis the National General 
Accreditation Commission and in international organizations. 
Finally, it is expected to speak out publicly on issues of 
accreditation and the ranking of higher education institutions. 

2.11. ROMANIA 

Romania has fifty-seven public and eighty-three private higher 
education institutions. Higher education institutions can be 
universities, academies, polytechnics, institutes, or colleges. 
Already in 1990, Romania had 101 private faculties, a number 
that more than doubled in the next decade, reaching 221 
faculties by 2000, with over 30 percent of students in higher 
education attending private institutions. With tuition fees 
being charged in public institutions, the private sector is highly 
competitive. However, accreditation is mandatory and very 
rigorous, and the 1993 law on accreditation requires, among 
other things, that at least 51 percent of students at a private 
institution pass the state examination at a public university. 
By 2001, only eight private higher education institutions had 
been accredited, thus authorized to grant state-recognized 
degrees as well as to be eligible for state funding. 

The National Council for Academic Assessment and 
Accreditation has nineteen to twenty-one members appointed 
for four-year terms. They are nominated by higher education 
institutions, the list of which is published in the daily press so 
as to invite public challenges. From the resulting short-list, the 
Minister of Education proposes the final membership list to the 
cabinet for approval by the Parliament. The members elect a 
president, a vice-president, and a secretary on the proposal of 
the Minister. 

The Council sets up fifteen expert committees of seven to 
nine members, who are appointed for a four-year term. The 
expert committees cover the basic disciplines and include 
some foreign members. They may involve additional experts for 
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reviews who may be non-academics and/or foreigners. The 
Council has a full-time staff of fifteen professionals and ten 
administrators and coordinators. 

Standards and criteria for accreditation are set down in the 
1993 Law on Accreditation. Quality assurance encompasses 
ongoing internal evaluation of higher education institutions 
and external assessment. The latter begins with the licensing 
of new institutions, accreditation three years after licensing, 
and repeated assessment every five years. 

New programmes also must be initially licensed. Accreditation 
is awarded after several assessments. The final decision on 
accreditation is a parliamentary decision, if it concerns a new 
institution, and governmental, if it concerns a new programme. 
The periodic institutional assessment covers all study 
programmes. The institutions that do not meet the 
requirements are given a year to improve. 

The reports of the initial review team are confined to facts 
and are first given to the university for verification. The next 
stage of the report is written by the expert committee and is 
based on the self-evaluation report of the institution and the 
review team report. The final report is discussed and approved 
by the fuU Council and may include recommendations for the 
university. The report is then submitted to the Minister who 
may return it for review to the commission. The report of the 
Commission and the comments of the Minister are sent to the 
institution, and if the latter agrees, may be made public. The 
final decision on accreditation is made by the Government (for 
programmes at accredited institutions) or by the Parliament 
(for new institutions). 

2. 12. THE RUSSIAN FEDERATION ? 

The Russian Federation, embracing fifteen post-Soviet states 
with 89 regions, has over 1,100 higher education institutions. 
Of these, 431 are licensed non-state institutions providing 9 
percent of the student population. State institutions were 
founded before 1990. There are also municipal and private 
institutions that were founded after that date. Higher 
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education is offered at four levels: vocational, non-university, 
university, and postgraduate, including Doctoral level. 

Quality assurance in Russia is based on three consecutive 
procedures, licensing, state accreditation, and attestation 
based on state standards. The process was made the object of 
legislation in the 1992 Law on Education. State standards 
describe minimum programme content, workload, and 
requirements. Licensing, granted by state or local authorities, 
gives institutions the right to offer study programmes in the 
field and at the levels for which licensing is granted. The 
experts check whether or not the conditions for education have 
been met, whether the institution and its programmes meet 
legislative and infrastructural requirements, and whether staff 
qualifications are adequate. Licensing is an administrative 
process conducted by external experts and federal and local 
educational authorities. 

Accreditation is the legislated product of a decree of the 
State Committee for Higher Education. It confers on both state 
and private institutions the right to issue state-recognized 
degrees. Accreditation testifies type of institution (professional 
school, technical school [also humanities and social sciences], 
college, institute, academy, university) and its level (general 
education, vocational education, non-university or university 
higher education), and whether or not the contents and quality 
of training comply with the State Educational Standards. The 
process of accreditation combines institutional and programme 
assessment and extends to all the programmes of the 
evaluated institution. Accreditation is granted for a maximum 
of five years for institutions and programmes and encompasses 
self-evaluation by higher education institutions and an 
external peer review. The review team, made up of academics 
and non-academics, writes a report, which forms the basis for 
accreditation. Non-universities are accredited by state 
authorities in the region in which the institution is located. 
Universities are accredited by the Ministry of General and 
Professional Education. 
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Attestation is the task of an expert commission, which also 
include non-academics. Its purpose is to assess the knowledge 
of graduates as compared to the State Educational Standards, 
and is regulated by the 1994 decree of the State committee for 
Higher Education on State Final Attestation of Graduates of 
Higher Education Institutions. Attestation is a rigorous exit 
examination procedure for graduates, which includes a final 
examination in the graduate’s discipline, a final examination in 
the specialization, and the defense of a thesis or project, 
conducted by a state attestation commission, involving 
academics and non-academics, set up by the body responsible 
for accreditation. Attestation is a precondition for the award of 
individual degrees as weU as part of the process of 
accreditation at the institutional level. 

The focus of the National Accreditation Center is to provide 
information to the Accreditation Council. The Council, 
established in 1997, is a unit of the Ministry of Education. It 
has forty-two members drawn from the Ministry and from 
various educational and non-state organizations. It organizes 
reviews and invites review teams. Review teams evaluate the 
institution and all its programmes and consist of five to fifteen 
experts in different fields, including management, education, 
and research. The role of the center is to compile 
documentation for the Ministry that makes accreditation 
decisions based on the report of the Accreditation Council. 

The Center works with a staff of about fifty full-time and 
about twenty part-time employees. The permanent staff of the 
Center includes five academics and eight top managers with 
one director and three deputy directors. There are six 
departments at the headquarters In Yoshkar-Ola and one 
branch in Moscow. A major part of the work of the Center is 
to compile data on higher education institutions, their 
programmes, and the accreditation status of both. The 
Center also conducts research Into accreditation and quality 
assessment. It designs assessment procedures and develops 
computer software for different stages of quality assessment. 
Another important service offered by the Center is to provide 
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information on accreditation status and procedures. This 
information can be obtained via the Website of the Center 
<http://www.nica.ru>. The Center staff includes academics 
and professionals with academic degrees. 

2.13. SLOVAKIA 

Slovakia has twenty- three higher education institutions, of 
which twenty-two offer Master’s Degree level and Doctoral 
education, while one private institution leads only up to the 
Bachelor’s Degree level. Ten of the higher education institutions, 
including the newly established Catholic university, are 
universities; three are technical universities with a range of 
fields; and three are professionally oriented universities. There 
are also three university-level academies of art, one police, and 
two militaiy academies. The latter three are considered “state” 
institutions, while the other non-private ones are “public” 
institutions. All higher education institutions, whether public, 
state, or private, must have state recognition to operate. 

More students apply for admission to higher education 
institutions than there are study places available. At the same 
time, institutions compete for students in less fashionable 
fields. 

The first Slovak Law on Higher Education following the 
1989 change of regime was introduced in 1990. A new Higher 
Education Act went into effect on 1 April 2002 reflecting two 
fundamental changes. Whereas in the past faculties were legal 
entities, nowadays it is the university that is a legal entity. It 
retains ownership of its properties, for the oversight of which, 
and some other strategic decisions, it must set up a Board of 
Trustees. The 2002 Law has changed some of the 
responsibilities of the Accreditation Commission. Moreover, the 
Minister of Education must now approve its regulations and 
standards. 

The Commission has twenty-one members, including 
foreign ones, who are appointed by government on the 
proposal of the Minister following consultations with the higher 
education institutions, the research institutes of the Academy 
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of Sciences, and other bodies, the latter possibly being 
recommended by various ministries. The Commission 
evaluates the capacity of institutions to implement 
programmes, including doctoral programmes, in non-higher 
education institutions, and to conduct habilitations and to 
nominate professors. In addition, it issues statements relative 
to proposed changes in a higher education institution [e.g., on 
creating a new faculty), about the setting up of institutions, 
about defining its type (university, non-university, or research 
university), about changing the structure of the study fields of 
given institutions, and about conducting the “complex” 
accreditation of an institution. It acts as an advisory body to 
government. 

There is a distinct separation between programme and 
institutional accreditation. Programme accreditation grants the 
institution the right to award degrees at the level specified and 
to habilitate and nominate professors according to the 
procedures that are also accredited. If the accreditation is 
positive, it may be granted without a time limit, but will be re- 
assessed in the context of “complex” institutional 
accreditation. If problems were identified, accreditation is given 
for one or two years, depending on the severity of the problems 
in question. The evaluation aspect in this process is negligible, 
and accreditation is based on direshold criteria. 

“Complex” institutional accreditation is conducted at six- 
year intervals. It encompasses the institutional level, including 
the procedures for habilitation and nomination of professors, 
as well as all programmes. The procedure should focus on 
improvement, with the Commission pointing out strengths and 
weaknesses. As part of complex institutional accreditation, the 
Commission also evaluates the performance of a given higher 
education institution in research and development (except in 
fields of art). As per the new law, the quality of research is no 
longer graded. 

It is interesting to note that the same procedure for 
evaluating research is applied to the Academy of Sciences as 
weU as to state and private research institutes. This fact 
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implies that all of these institutions will vie for research funds 
from the state budget on the basis of accreditation. 

The decisions of the Commission are formulated as 
opinions addressed to the Minister, or, in the case of certain 
specific tasks, to the Government or Parliament. However, if 
the decision of the Minister regarding accreditation is different 
from that of the Commission, he or she must, by law, publish 
his or her reasons. 

Currently, the technical staff of the Accreditation 
Commission consists of one academic-professional person and 
one administrator. A government decree will lay down the 
basic conditions for operation. 

2.14. SLOVENIA 

In 1993, Slovenia had two comprehensive universities and two 
“free-standing professional higher education institutions”. 
Today, the number of universities remains unchanged (with, 
altogether, thirty-seven faculties), but there are also seven 
private higher education institutions. The “free-standing” 
schools are required to have state recognition. 

The Higher Education Act of 1993 set up a Council for 
Higher Education. Its standards are specified in the Act, and 
its criteria for evaluation, in its “Criteria and Procedures on 
Accreditation of Study Programmes and Higher Education 
Institutions” (in, Zgaga and Jurkovic, 1995). 

Six of its fourteen members and its President are appointed 
by the Government, after nomination by the higher education 
institutions, for four-year terms. They are joined by the 
President of the Academy of Sciences and Arts and the rectors 
of the universities. 

Among the responsibilities of the Council are those of 
establishing the criteria for assessing the quality of teaching 
and with approving study programmes. It is also expected to 
determine whether or not an institution is fulfilling the legal 
requirements to be established and operated, and whether new 
study programmes can be set up, as a precondition for 
funding. The criteria for assessing higher education 



QUALITY ASSURANCE 1 1 5 



institutions are also set down in the “Criteria and Procedures”. 
Moreover, the Council is responsible for judging whether or not 
a faculty member may be awarded a title, based on the 
requirements of the law and the opinion of external experts. 

In 1996, the higher education institutions took the initiative 
to establish the Quality Assessment Commission. It was 
restructured in 2000. Its main purpose is to develop a self- 
evaluation methodology and to assist higher education 
institutions in setting up their internal quality control 
mechanisms. The Commission basically collects and 
disseminates information about evaluation at institutions. The 
Commission has eighteen members and a president. There is 
only one administrator and no programme officers. 

The overall tasks of the Commission are defined in the 1997 
amendment to the Higher Education Act as follows: 

The quality and effectiveness of teaching, research, art, and 
professional activities of higher education institutions shall be 
monitored and assessed by a Quality Assessment Commission 
created by higher education institutions in the Republic of 
Slovenia. The Commission shall be composed of representatives 
of all scientific and art disciplines and professional fields. The 
Commission shall also obtain the advice of students. The 
Commission shall conduct business according to the rules 
deterrnined in co-operation with the senates of higher 
education institutions and criteria defined by the Council of 
Higher education in co-operation with the Council for Research 
and Technology. Once a year, the Commission shall report to 
the Senates of the higher education institutions, the Council for 
Higher education, and the Council for Research and 
technology. The report shall be publicly dissemiaated (Article 
89, Higher Education Act of the Republic of Slovenia, amended 
in 1997). 

With more students applying than there are places available 
for them, at least in some areas of study, there is competition 
among higher education institutions for students. Accreditation, 
however, is not yet mandatory. 

The law in Slovenia has, in effect, established two bodies, 
one, which answers to the Government, and another, which 
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answers to the higher education institutions (but, as a 
nationally legislated body, also to government), with the two 
expected to share criteria. With initially only a handful of 
higher education institutions the sector felt little pressure to 
organize its system of quality assurance. The sharp rise in the 
number of institutions in the last two to three years is bound 
to speed up the process. 

2.15. UKRAINE 

In 2000, Ukraine had 809 state and 161 private higher 
education institutions. The Law on Education of the country 
was passed by Parliament in 1991 and amended in 1996, 
establishing the broad legislative framework for education. The 
elaboration of a higher education law began in 1996; however, 
no such law has yet been passed. In line with the ongoing 
reform, the Ministry of Education and Science was reorganized 
in 1997. It is responsible for over half the state higher 
education institutions, while the rest are under other 
ministries. The respective ministries are involved in setting the 
educational content and the accreditation of the higher 
education institutions under their authority, while the 
education ministry retains responsibility for the education 
process. It is also in charge of conducting the accreditation of 
higher education institutions, setting admission standards, 
and issuing licenses and certificates. The autonomous 
Republic of Crimea has a separate Ministry of Education, but 
it abides by the regulations of the Ukrainian ministry. A State 
Accreditation Commission, as a unit of the Ministry, is 
responsible for carrying out the accreditation procedures. 
Accreditation was introduced with the 1992 law and is 
currently regulated according to a Regulation of the Cabinet of 
Ministers (No. 200, 12 February 1996). 

The accreditation of higher education institutions is 
awarded up to one of four levels pertaining to the institutional 
type and diploma or degree offered. Level I is for technical and 
vocational schools leading to the certificate of “junior 
specialist”. Level II is for colleges leading to a Bachelor’s 
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Degree. Levels II and IV are for institutes, conservatories, 
academies, and universities leading to “Specialist” and 
Master’s Degrees. 

The level of accreditation determines the degree of 
autonomy of the institution as well as its funding, with only 
the highest level receiving full financial support. Of the 809 
state higher education institutions in 2000, 220 were of Levels 
III and rv, while 589 were of Levels I or II. Of the 161 non-state 
higher education Institutions, only forty-one were fully 
accredited. A shift to the higher levels is apparent (from 161 in 
1995 to 220 in 2000). 

New higher education institutions are first licensed by the 
State Accreditation Commission. Licensum does not represent 
a qualitative judgment but merely allows the institution to 
begin operations. The succeeding step is accreditation, which 
gives the institution the right to issue state-recognized degrees, 
issued after completion of education based on national 
standards. A course programme can be submitted for 
accreditation only after it has been offered for five years. 
Accreditation covers all study programmes and must be 
renewed every ten years. Institutions are awarded the level of 
accreditation that has been awarded to 75 percent of its 
programmes. 

The Commission is regulated by the Constitution, by 
legislation passed by the Parliament, by the President of 
Ukraine, by the Cabinet of Ministers, and by the Ministry of 
Education. The Commission sets the criteria for accreditation 
and publishes accreditation results. The members of the 
Commission, including civil servants from the education and 
other ministries, as well as state executives, academics, and 
researchers, must be confirmed by the cabinet of Ministers. 
The Minister is the Chairperson of the Commission. The 
Commission has a network of local and regional expert 
councils who report on the individual programmes submitted 
for accreditation. The Commission adopts the final decision 
relative to the level of accreditation awarded. 
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2. 16. COUNTRY OVERVIEW OF ACCREDITATION AND 

QUALITY ASSESSMENT AGENCIES 

2.16.1. Clcirification of Terminology 

Name of Agency/Commission, Year Established: tJie official 
name of t±ie agency or commission at present and the 
year the agency or commission was set up. 

Initiative to Estabush Agency/Commission: either the 

government or a group of higher education institutions 
which joined to initiate quality assurance in the country 
or for members of the group. 

Ownership of Agency/Commission: To whom does the agency 
and/ or commission report? 

Membership, Staff: Who delegates/names, the members of the 
commission, their number, and their background; how 
many staff members and their role? 

Scope of Activities, Orientation: What are the main tasks of 
the commission/agency, what types of 
assessment/accreditation do they conduct, and the 
objective of the assessment? 

Cycle of Assessment: For how many years is accreditation 
valid; how often is assessment repeated? 

Methods: The common approach would be self-evaluation, 
peer review, a report, but other methods may be 
possible. 

Grading: Is the evaluated institution or programme given a 
quality mark on a scale used by the agency/ commission, 
or are there different levels of accreditation, yes or no? 

Criteria FOR Assessment; Who develops the criteria, standards, 
and methods to conduct assessment? 

Review Team Training or Briefing: Which or neither, perhaps 
only written information? 

Foreign Commission Members, Reviewers: Are foreign 

nationals members of the commission and /or are foreign 
nationals invited to participate in the evaluation; if yes, 
which of them? 

Report Published: Yes or no? Clarification if needed. 
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2.16.2. Schernatic Presentation of Accreditation and Quality 
Assessment Agencies by Central and Eastern European Countries 

Albania 



^ , . . X i-i- u j Accreditation Agency of Higher Ekiucation 

Name of agency /commission, year established Accreditation Council. 1999 



Initiative to establish agency/ commission 
Ownership of agency/ commission 

Membership, staff 

Scope of activities, orientation 
Cycle of assessment 

Methods 

Grading 

Criteria for assessment 

Review team training or briefing 
Foreign commission members, reviewers 
Report published 



Government 

Government (Independent decisions on 
quality but advisory to minister) 

1 1 Council members(academics); 5 Agency 
staff with collaborating, coordinating, 
administrative role 
Institutional and programme-level 
accreditation. Improvement oriented 
4 years 

Self-evaluation according to Agency 
guidelines, review team with on-site visit 
after agency guidelines 
Yes 

Set by Agency and expert team based on 

legislation 

Training 

No 

Yes 



Bulgaria 



Name of agency /commission, year established 
Initiative to establish agency/ commission 
Ownership of agency /commission 

Membership, stcff 

Scope of activities, orientation 

Cycle of assessment 

Methods 

Grading 

Criteria for assessment 

Review team training or briefing 
Foreign commission members, reviewers 
Report published 



National Evaluation and Accreditation 
Agency, and Accreditation Council, 1996 
Government 

Government (Independent decisions on 
quality but advisory to minister) 

9 Council members (academics), 20 
Agency staff with coordinating and 
administrative role 

Institutional, subsequently programme 
accreditation, with accountability and 
improvement orientation 
5 years, 1-3 years if assessed as 
“Satisfactory” 

Self-evaluation according to Agency 
guidelines, review team with on-site visit 
according to Agency guidelines 
Yes 

Criteria set down by Agency based on 

legislation 

Briefing 

Council no, review team sometimes 
Yes 



BEST COPY AVAILABLE 
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Croatia 



Name of agency /commission, year established 


National Council for Higher Education , 1993, 
1999 


Initiative to establish agency/ commission 


Government 


Ownership of agenqj/ commission 


Government (unit in, and advisory to. 
Ministry, which organizes reviews) 


Membership, staff 


19 Council members (academics), 3 full-time, 
1 part-time staff in Ministry department 


Scope of activities, orientation 


Evaluation of institutions, accreditation of 
programmes, with improvement orientation 


Cycle of assessment 


Not set 


Methods 


Self-evaluation according to Council 
guidelines, review team with on-site visit 
according to Council guidelines 


Grading 


Yes 


Criteria for assessment 


Criteria set down by Council based on 
legislation 


Review team training or briefing 


Briefing 


Foreign commission members, reviewers 


Council no. Review teams yes 


Report published 


No 


The Czech Repubuc 


Name of agency /commission, year established 


Accreditation Commission, 1990 


Initiative to establish agency/ commission 


Government 


Ownership of agency /cornrrdssion 


Government (independent decisions on 
quality but advisory to Minister) 

21. members (11 academics and 10 


Membership, staff 


professional experts): 6 staff with 
coordinating and administrative role 


Scope of activities, orientation 


Accreditation of programmes and institutions. 
Improvement orientated 


Cycle of assessment 


Max. 2 X length of programme (doctoral max. 
10 years). 


Methods 


Institutions have independent internal 
evaluation. Self-evaluation according to 
agency guidelines, review team with on-site 
visit according to agency guidelines 




Grading 


Yes (Le., 3 levels of negative decision) 


Criteria for assessment 


Elaborated by review team based on 
Commission criteria and legislation 


Review team training or briefing 


Written information 


Foreign commission members, reviewers 


Yes 


Report published 


Yes, both internal and external evaluation 
reports 
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Estonia 



Name of agency /commission, year established 

Initiative to establish agency/ commission 
Ownership of agency /commission 

Membership, stcff 

Scope of activities, orientation 

Cycle of assessment 

Methods 

Grading 

Criteria for assessment 
Review team training or briefing 
Foreign commission members, reviewers 

Report published 



Higher Elducation Accreditation Center, 
Archimedes Foundation, 1997: Higher 
E^ducation Quality Assessment Council, 1995 
Government 

Government (Council); foundation (Center) 

12 Council members (academics): 3 Center 
staff with coordinating and administrative 
role 

Programme, some institutional accreditation. 
Improvement oriented (separate research 
evaluation) 

7 years for full, 2 years for conditional 
accreditation 

Self-evaluation according to Agency 
guidelines, review team with on-site visit 
according to Agency guidelines 
Yes 

Determined by Council based on legislation 
and approved by Minister 
Written information 

In Council no. Review team only foreign 
members 

Not broadly but sent to Institutions and to 
student organizations 



Hungary 



Name of cujency/ commission, i/ear established 


Hungarian Accreditation Committee, 1993 


Initiative to establish agency/ commission 


Government 


Ownership of agency/ commission 


Government 




30 full members (academics and 
professional body representatives), non- 


Membership, staff 


voting members to cover spectrum of 
disciplines, 1 non-voting student: 26 staff 




with collaborating, coordinating, 
administrative role 


Scope of activities, orientation 


Accreditation of institutions including its 
programmes. Improvement oriented 


Cycle of assessment 


8 years 

Self-evaluation according to agency 


Methods 


guidelines, review team with on-site visit 
according to agency guidelines 


Grading 


Yes 


Criteria for assessment 


Set by agency, with national qualification 
requirements legislated 


Review team training or briefing 


Briefing 


Foreign commission members, reviewers 


Sometimes 


Report published 


Yes 
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Latvia 

Higher Education Quality Evaluation Center 

Name of agenqj/ commission, year established Ltd.. 1994. Council for Higher Education. 

1995. Accreditation Commission. 1995 

Initiative to establish agency! commission Government 

Ministry and 5 higher education institutions 
are shareholders of agency 
6 agency staff with coordinating, 
administrative role; 10 member Accreditation 
Commission; 10 member Council (academics 
and non-academics) 

Institutional and programme accreditation 
with improvement orientation 
Permanent or 2 years for Institutions. 6 years 
(or 2 years only once) for programmes 
Self-assessment based on agency guidelines, 
site-visit based on methodology by team 
Recommended for programmes but rarely 
used 

Established by agency based on legislation 
Written information, sometimes training or 
briefing 

^ . ... Yes (except for review of college-level 

Foreign commission members, reviewers vocational higher education institutions) 

Report published Yes 



Ownership of agency /commission 

Membership, staff 

Scope of activities, orientation 
Cycle of assessment 
Methods 
Grading 

Criteria for assessment 
Review team training or briefing 



Lithuania 



Name of agency /corruriissiDn, year established 



Initiative to establish agency/ commission 
Ownership of agency /commission 
Membership, staff 



Scope of activities, orientation 



Cycle of assessment 



Methods , 

Grading 

Criteria for assessment 
Review team training or briefing 
Foreign commission members, reviewers 
Report published 



Lithuanian Center for Quality Assessment 

in Higher Education. 1995 

Ministry- of Education and Science of the 
Republic of Lithuania; Lithuanian Science 
Council 
Government 

Expert evaluation division head (academic, 
part-time) + 3 full-time agency staff with 
coordinating role 

Institutional and programme assessment. 
Accreditation by Minister. (Separate 
research assessment also). Improvement 
orientation 

Not set. 8 years for first cycle for all study 
programmes, reassessment within 2 years 
after temporary or restricted assessment 
Self-evaluation according to agency 
guidelines, review team with on-site visit 
according to agency guidelines 
Foreseen 

Established by agency based on legislation 
Workshops and briefing 
Review teams for specific subject areas 
Reviews yes 
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Macedonia 



Name of agency /commission, year established 


Elvaluation Agency, and Accreditation Board, 
2000 


Initiative to establish agency/ commission 


Inter-University Conference 


Ownership of agency /commission 


Government 

9 Agency members with coordinating. 


Membership, staff 


administrative role; 15 Board members 
(academics) 


Scope of activities, orientation 


Institutional and programme accreditation. 
Improvement orientation 


Cycle of assessment 


Five years 

Self-evaluation according to Agency 


Methods 


guidelines, review team with on-site visit 
according to Agency guidelines 


Grading 


No 


Criteria for assessment 


Established by Board and by Agency-based 
on legislation 


Review team training or briefing 


Written information 


Foreign commission members, reviewers 


Sometimes 


Report published 


Yes 



Poland 

,r „ i ui- L. j University Accreditation Commission (UAC), 

Name of agency/ commission, year established , 



Initiative to establish agency/ commission 
Ownership of agency /commission 

Membership, staff 

Scope of activities, orientation 

Cycle of assessment 

Methods 

Grading 

Criteria for assessment 

Review team training or briefing 
Foreign commission members, reviewers 
Report published 



Rectors Conference 
Non-govemmental 

18 members (academics, Le., 17 vice-rectors 
or representatives of signatory universities + 

1 Rectors Conference delegate): 2 full-time 
and 1 half-time staff, coordinating, 
administrative role 

Accreditation of member and non-member 
university prograiximes at research 
universities, voluntary. Improvement 
orientation 
2-5 years 

Self-study based on agency guidelines, 
external review with on-site visit 
No 

Standards set by own expert committee, 
reviewed every two years, methodology 
worked out by commission 

Briefing 

Review team no, expert committee sometimes 
No, only for evaluated unit 
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Poland (cont) 



f / ' * ^ uf u ^ University Accreditation Commission (UAC), 

Name of agency /commission, year established ^ 



Name of agency /commission, year established 

Initiative to establish agency/ commission 
Ownership of agency /commission 

Membership, staff 

Scope of activities, orientation 

Name of agency /commission, year established 
Name of agency /commission, year established 

Name of agency /cornrnission, year established 

Name of agency /cornrnission, year established 
Name of agency /commission, year established 
Name cf agency /commission, year established 
Name of agency /commission, year established 

Name of agency /commission, year established 

Name of agency /commission, year established 

Initiative to establish agency/ commission 
Ownership of agency /cornrnission 

Membership 



National General Accreditation Commission, 
January 2002. Expected to succeed General 
Council on Higher Eklucation, 2000 (Law: 
1990) 

Government 

Government (unit in ministry) 



Intention was to evaluate without formal 
accreditation at institutional and programme 
level, accountability, some improvement 
orientation (The General Council had a 
number of other tasks not directly related to 
quality control) 

Business Schools for Quality of Higher 
Education (SEM FORUM) (private), 1993 

Accreditation Committee of Medical Schools 
(State), 1997 

Accreditation Commission of Schools of 
Higher Vocational Education (State and 
private), 1997 

Accreditation Commission of Schools of 
Physical Education (State), 1999 

Accreditation Commission of Higher 
Pedagogical Schools (State), 1999 

Accreditation Commission for Technical 
Universities (State), 2001 

Accreditation Commission for Agricultural 
Universities (State), 2001 

Accreditation Commission of the Foundation 
for Promotion and Accreditation of Ek:onomic 
Studies (State), 2001 

Accreditation Commission of the Rectors of 
Polish Academic Schools, 2001 

Rectors Conference 
Non-govemmental 

The 7 state academic accreditation 
commission chairpersons + a president , an 
umbrella organization 
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Romania 


Name of agency /commission, year established 


National Council for Academic Assessment 
and Accreditation, 1994 


Initiative to establish agency/ commission 
Ownership of agency /commission 

Membership, staff 

Scope of activities, orientation 
Cycle of assessment 

Methods 

Grading 

Criteria for assessment 

Review team training or briefing 
Foreign commission members, reviewers 
Report published 


Government 

Parliament 

19-21 council members (academics); 25 
staff with coordinating and administrative 
role 

Institutional and programme accreditation 
with improvement orientation 
5 years 

Self-evaluation according to agency 
guidelines, review team with on-site visit 
after agency guidelines 
No 

Established by expert commissions based 
on legislation ^ 

Training 

Reviewers sometimes 
If institution agrees 



The Russian Federation 





National Accreditation Center, 1995; 


Name of agency /commission, year established 


Accreditation Council (Ministry for General 
and Professional Education), 1997 


Initiative to establish agency/ commission 


Government 


Ownership of agency /commission 


Government 

50 fuU-tlme and 20 part-time Center staff; 42 


Membership, stcfff 


Council members (academics and non- 
academics) 


Scope of acttvtties, orientation 


Institutional and programme accreditation. 
Improvement orientation 


Cycle of assessment 


Maximum 5 years for institution or 
programme 


Methods 


Developed by ministry 


Grading 


No 


Criteria for assessment 


Developed by evaluation team, based on 
legislation 


Review team training or briefing 


Written information 


Foreign commission members, reviewers 


No 


Report published 


Yes but not widely distributed 
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Slovakia 



Name of openaj/ commission, year established 


Accreditation Commission. 1990 


Initiative to establish agency 1 commission 


Government 


Ownership of agency /convnission 


Government 

27 members{academlcs): old commission had 


Membership, staff 


2 staff members with coordinating role and 
head contributes to report 


Scope of activities, orientation 


Institutional and programme accreditation, 
improvement orientation 


Cycle of assessment 


Programme accreditation without time limit, 
but 1-2 yeeirs if shortcomings observed, 6 
years for institutional accreditation 


Methods 


Self-study based on agency guidelines, 
external review with on-site visit based on 




agency guidelines 


Grading 


No 


Criteria for assessment 


E^stablished in law and by commission 


Review team training or briefing 


Briefing 


Foreign <x>rnrnission members, reviewers 


Commission yes, review team sometimes 


Report published 


No 


Slovenia 


Name of agency /<x>rnjrdssion, year established 


Council for Higher Eklucation, 1994 


Initiative to establish agency! commission 


Government 


Ownership of agency /commission 


Government + Rectors Conference 


Membership, staff 


15 (academics) 


Scope of activities, orientation 


A government advisory body. Institutional 
and programme accreditation 


Cycle of assessment 


Not set 


Methods 


Self-study, external review with on-site visit 
based on agency guidelines 


Grading 


Yes 


Criteria for assessment 


Law + agency 


Review team training or briefing 


No 


Foreign commission members, reviewers 


No 


Report published 


Only decision and summary reason with 
negative decisions 


Name of agency/ commission, year established 


Quality Assessment Commission of Slovenia, 
1996, reorganized 2000 


Initiative to establish agency/ commission 


Rectors Conference 


Ownership of agency /commission 


Government 


Membership, staff 


19 (academics + 2 students), 1 administrator 
No external evaluation, but advisory to 


Scope of activities, orientation 


Council for Higher Eklucation. Improvement 
oriented 
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Slovenia (cont) 


Name of agency/commission, year established 


Council for Higher Education, 1994 


Cycle of assessment 

Methods 

Grading 

Criteria for assessment 

Review team training or briefing 
Foreign commission members, reviewers 
Report published 


Not set 

Self-study based on institution’s own 
approach, external review with on-site visit 
serves to advise on self-evaluation 
No 

Agency, according to standards by Council of 

HE 

No 

No 

No 


Ukraine 


Name of agency /commission, year established 


State Accreditation Commission 
(Ministry of Education and Science), 1992 


Initiative to establish agency/ commission 
Ownership of agency/ commission 

Membership, staff 

Scope of activities, orientation 
Cycle of assessment 
Methods 
Grading 

Criteria for assessment 
Review team training or briefing 
Foreign commission members, reviewers 
Report published 


Government 

Government 

\ 

Members from various ministries, academics, 
researchers, state executives, chaired by 
education minister 

Programme and institutional accreditation 
10 years for programme accreditation 
Set by commission based on legislation 
No data 

Set by commission based on legislation 
No data 
No 

Outcomes yes 



2.16.3. Internet Links for Quality Assurance in the Centred 
and Eastern European Countries 

- <ht±p: / /www.ceenetwork.hu> 

- individual agency links (access from 
<http://www.ceenetwork.hu>) 

- information on CEE agencies also at 
<http://www.inqaahe.nl> 
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2.16.4. Questionnaire Sent to Quality Assurance Agencies in 

Fifteen Central and Eastern European Countries 

a) Could you please send me a bibliography on quality 
assurance and your schools’ systems to list in the 
bibliography section of the book; 

b) How many higher education institutions in the country? 
Number of universities: Number of others? Number of state 
institutions? Number of private institutions? Do they need 
state recognition to operate? 

c) Do you see QA as a tool for controlling the proliferation of 
private institutions? 

d) Do more students apply to higher education institutions 
than there is room for them? 

e) Do you feel that there is competition among higher 
education institutions to enroll students? If yes, is 
evaluation/accreditation a publicly recognized factor for 
choosing a higher education institution? 

J) Who is involved in making up study programmes, the 
higher education institution, the ministry, your agenc}^ 

g) Scope of your agency: accreditation, evaluation, or both? If 
accreditation, for institutions, or programmes, or both? 

h) Do you evaluate institutions and/or programmes? If both, 
in separate procedures or in a single procedure? Is 
evaluation mandatory or voluntary? 

i) By whom is the accreditation/evaluation commission 
nominated/appointed? By government [te., minister, prime 
minister) or higher education institutions [te., rectors 
conference)? 

J) What is the size of office staff and what are the functions of 
the members (programme officers, administrators)? If you 
have programme officers, do they have coordinating roles, 
participate in visits, contribute to reports? 

k) Do you use a grading scale (such as Excellent, Good, 
Satisfactory...) in your assessment? If yes, for institutional 
and/or programme evaluation? 

l) Who devises the methodology and criteria for evaluation? 
Who devises questions for the institutional self-evaluation? 
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What is the ratio of data requested and of the anal5dlcal 
part? How long is the self-evaluation? Are higher education 
institutions consulted? 

m) What is the aim of the evaluation: accreditation? fitness of 
purpose? improvement? compliance with laws? (more than 
one possible)? Does your agency advise the Minister and 
how binding is a decision by your commission (does the 
Minister have to publish his reasons for not agreeing with 
the Commission)? 

n) Do you prepare or train the review team? How? 

o) Are its reports published? How detailed is your report 
(anything from “complies with the law” to detailed 
description of strengths and weaknesses)? 

p) Do you include foreign members in review teams? 
Sometimes? Always? 

q) Who decides on the schedule of the visit? When to visit an 
institution and/or a programme and the make-up of the 
review team? For how many years is the evaluation/ 
accreditation valid? 

The authors hereby wish to thank the contact persons of 

the CEE Network agencies, who contributed considerable time 

and effort to reply to the questionnaire and to do even more. 
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Annex 3 



Glossary 



The definitions appearing below have been derived from a 
number of sources. In particular, those marked with a single 
asterisk (*) are based on definitions appearing in Quolity 
Assurance in Higher Education - Manual of Quality Assurance 
in Higher Education: Procedures and Practices (Turin: European 
Training Foundation, 1998, p. 12). Those mairked with a 
double asterisk (**) aire derived from the questionnaire for a 
survey on quality assurance agencies conducted, in December 
2001, by the International Network of Quality Assurance 
Agencies in Higher Education (INQAAHE). See 
<h ttp : / / WWW. inqaahe . nl> . 

AccountahHity: The assurance of a unit to its stakeholders that 
it provides education of good quality. 

Accreditation: The award of a status which signals approval, 
recognition, and sometimes a license to operate and is 
based on pre-defined standards*. It is a formal, yes/no 
(sometimes also conditional) decision based on explicit 
minimum (threshold) requirements**. 

Agency: Organization or office responsible for preparing, co- 
ordinating, and carrying out evaluation or accreditation 
procedures but not actually taking decisions on 
outcomes. The term, agency, is often used collectively, 
when in fact, a committee passes the final evaluation 
decisions. 

Benchmark: A reference point or criterion by which to measure 
something. 

Benchmarking: Setting levels against which quality is 

measured or a process of identifying and learning from 
good practice in other organizations. 

Binary (or dual) system: When higher education in a country is 
offered in two separate and distinct types of institutions - 
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universities and non-university establishments (cf., 
unitary system). 

Criteria: “Checkpoints, the benchmarks for assessing the 
quality of the input and process**. Often used 
synonymously with “standards”. 

Evaluation: Any process leading to judgments and/or 

recommendations regarding the quality of a unit*. 

Evaluation team: Same as a review team. 

Ex-ante assessment/ evaluation: Assessing/evaluating quality 
before a programme or institution is launched, often as a 
condition for granting a license to operate. 

Ex-post assessment/ evaluation: Assessing/ evaluating quality 
after a programme or institution has been in operation in 
order to establish strengths and weaknesses. 

External evaluation: Evaluation of the quality of a unit carried 
out by a selected team of experts who are not connected 
to the unit evaluated. 

External expert Also: peer inspector. Member of a selected 
team evaluating a unit but who is not linked to it. 

Fitness for purpose: One of the possible criteria for establishing 
whether or not a unit meets quality, measured against 
what is seen to be the goal of the unit. 

Higher education: Part of tertiaiy education leading to a degree 
or equivalent diploma. 

Improvement orientation: Evaluation or assessment describing 
the strengths and weaknesses of a unit to facilitate 
improvement, and possibly including suggestions as to 
how to achieve it. 

Inspector. Also: peer, external expert. Member of a selected 
team evaluating a unit but who is not connected to it. 

Internal evaluation: Same as self-evaluation, done by a unit as 
a form of quality management or in preparation for 
external evaluation. 
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One-level degree structure: Also: one-tier, or one-cycle, whereby 
one course of studies leads to one level of degree (either 
in unitary or binary systems) as opposed to a two-tier 
degree structure, which offers a shorter degree 
programme and a choice of post-graduate programmes. 

Licensing: Award to operate a new unit granted if it meets 
established criteria for quality. 

Peer. Also: external expert, inspector. Member of a selected 
team evaluating a unit but who is not connected to it. 

Programme officer. Upper-level employee of an evaluation or 
accreditation agency coordinating, assisting, and 
carrying out the adriimistration of evaluation or 
accreditation procedures. 

Quality assessment Same as quality review. Evaluation but 
including its external dimension*. The “evaluation of 
quality itself,”** either on the institutional or programme 
level. 

Quality assurance: Policies, processes, and actions to maintain 
quality. The focus is on accountability to the 
stakeholders* of a unit. 

Quality audit Evaluation of an institution’s processes for 
quality management*. 

Quality commission, committee: Standing body of experts 
chosen by government or a group of higher education 
institutions for a specified term to establish the quality of 
institutions or programmes and frequently also to state 
its opinion on matters concerning higher education. 

Quality control Like evaluation, but stressing the internal 
measurement of quality of a unit. Often used 
synon5nnously with quality management*. 

Quality control Aggregate of measures taken regularly to 
assure quality of a unit, but the emphasis is on assuring 
that a prescribed threshold of quality is met. 

Quality enhancement Procedures taken to improve quality. 
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Quality management Aggregate of measures taken regularly to 
assure quality of a unit. Emphasizes the goal to improve 
quality. 

Quality revieux. Same as quality assessment. Evaluation but 
including its external dimension*. 

Review team: Same as (external) evaluation team, expert 
committee, visiting team, or peer review team. A group of 
experts not linked to the evaluated institution, unit or 
programme, who report on its quality based on 
information provided by the unit and on-site visits. 

Self-evaluation: Same as internal evaluation, done by a unit as 
a form of quality management or in preparation for 
external evaluation. 

Staff: Employees of an evaluation or accreditation agency co- 
ordinating, assisting, and carrying out the adrninistration 
of evaluation or accreditation procedures. Staff may 
consist of programme officers with coordinating and 
assistance functions and of office assistants or 
administrators with secretarial functions. 

Stakeholder. Students, society, and government participating 
in or benefiting from the provision of education. 

Standards: Expected outcomes of education, competencies 
expected from graduates. May be general standards for a 
degree level [e.g., Bachelor’s or Master’s Degree) or 
subject specific standards.** Often used synon5miously 
wit.i “criteria”. 

(Subject) benchmark statement Represents general expectations 
about standards (levels of student attainment) at a given 
level in a particular subject area. 

Tertiary education: Any education entered after successful 
completion of secondary education, which may include 
vocational post-secondary education (leading to a 
certificate) and higher education (leading to a degree), 
even though the designation is often used s)mon5miously 
with higher education. 
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Unit An institution, a faculty, or programme of study*. 

Unitary system: Higher education in a country offered in one 
type of institution, consisting of universities or 
university-like establishments [cf., binary system). 



Annex 4 



Quality Assurance: Aspects of Processes, 
Approaches, and Networking 



There is no “one size fits all models” for a quality assurance 
system. Each existing system is adapted to the cultural, social, 
and political context in which it has been established. 
However, a small sample of information collated from 
published documents, project reports, workshop sessions, and 
Website, is included here to illustrate some aspects of external 
evaluation and current developments in quality assurance in 
Europe. The selection is limited by the availability of 
information in English and the ease of accessibility for follow- 
up as well as by the fact that many systems of quality 
assurance in higher education are, at the moment, in the 
process of development or change. Information has been 
included from quadity assurance systems in small countries, 
from sector specific agencies, those in which legislation is a 
factor in setting standards and those in which it is not. 
Inclusion is not an indicator of best practice, neither is 
exclusion an indicator of less good practice. 

Readers are advised to go to the Website of the networks for 
quality assurance (see Part III) and follow links to individual 
Quality Assurance and Accreditation agencies, where much 
more information, including details of contact persons, is 
available. While several agencies may not have their guidelines 
available on the Web they will make them available in hard 
copy on request. 

The authors would like to thank all of those who have 
provided information or have agreed to allow it to be made 
available. 



137 



138 



C. CAMPBELL and C. ROZSNYAI 



4.1. QUALITY ASSURANCE AGENCIES: PRINCIPLES, 

VALUES. AND PROCESS STANDARDS 

Some examples of statements concerning current and 

proposed principles, values, and process standards of Quality 

Assurance Agencies are presented below. 

4.1.1. FH Conference /FH Council Austria: External Evaluation of 

Fachhochschiden in Austria <http: / /www.fhr.ac.at/english> 

Principles 

1. The scientific evaluation pursuant to Section 13 
Paragraph 2 FH Studies Act as amended consists of a self- 
evaluation report of the FH study programme and the 
maintainer (provider), the peer report, and the statement 
of the maintainer. 

2. In the preparation of the self-evaluation report (abbr. SE 
Report), the division of responsibilities between the FH 
study programme and the maintainer has to be taken into 
account in view of the autonomy of the FH study 
programme pursuant to Section 12, Paragraph 2, Sub- 
paragraph 5, and Section 12, Paragraph 4, Sub-paragraph 
2, FH Studies Act as amended. 

3. Prerequisite for the preparation of the SE report is the 
existence of an internal quality management system that 
should already be installed and documented in outline at 
the time of preparation of the SE Report. 

4. The choice and design of the internal quality management 
system is the duty of each individual Fachhochschule 
study programme. 

5. The quality system, FC Conference/FH Council, is based 
on the internal self-evaluation by the Fachhochschule 
study programme and the maintainer in accordance with 
the prescribed general conditions of the fifteen sections. 
Based on this evaluation report, an external peer group 
examines the activities of the FH study programme and 
the maintainer. 
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6. TTie peer group consists of four peers and an assistant. 
The peer group elects a group leader who nominates an 
assistant. The four peers are nominated jointly by the FH 
Council and the FH Conference and are appointed by the 
FH Council. The peers are assigned by the maintainer of 
the FH study programme to be evaluated. 

The external evaluation takes place in a form that allows 
the assessment of the achievement of the goals and the 
fulfillment of the targets of the approved programme to be 
evaluated. 

8. The assessments and the recommendations of the peers, 
after examination by the FH Council for their compliance 
with the FH Studies Act and the decisions of the FH 
Council, can be taken into account in the maintainer’s 
application for extension of recognition. 

9. Hie evaluation procedure is mainly founded on the quality 
concept, “fitness for purpose”, that is, the quality of a 
study programme is measured by the degree to which the 
prescribed and its own objectives have been fulfilled. 

10. The main task of the evaluation is establishing and 
assessing the discrepancies between the prescribed aim or 
the intended quality of the application and the actual state 
of the FH programme. This does not rule out 
recommendations by the peers. 

4.1.2. Denmark: Danmarks Evcdueriiigsinsitut (EVA) 

<http :/ / www.euadJo 

Central Values 

All EVA evaluations are based on the principles of 
improvement, visibility, and professionalism. 

Improvement 

Evaluations must identify strengths and weaknesses and thus 
promote the improvement of teaching and learning based on 
the objectives of the field of education - not on already 
accepted standards. 
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Visibility 

E^valuations must make the quality of teaching and learning 
visible, highlighting what is akeady successful and what may 
be improved. 

Professionalism 

Evaluations must be based on the expertise already available 
within the educational sector concerned. For this reason, 
experts familiar with the specialized field are always invited to 
participate in the evaluation. Evaluation officers from EVA 
contribute to the evaluation know-how. 

4.1.3. Germany: Accreditation, Certification, and Quality 
Assurance Institute (ACQUIN) <http:/ / www.acquin.org/english> 

The main characteristics of ACQUIN are independence, 
objectivity, and high quality. The agency is oriented towards 
the realization of academic principles and is neither influenced 
by the state nor by lobbyists. Academic freedom and academic 
autonomy are respected: higher education institutions may 
regulate their own quality and standards, but at the same time 
they must guarantee transparency of process and public 
accountability in discharging this self-regulation. 

4.1.4. The United Kingdom: The Quality Assurance Agency for 
Higher Education (QAA) <http://www.qaa.ac.uk> 

Draft Handbook for Institutionol Review: Annex J: The 
Agency's Operational Principles and Process Standards 

Background 

1. The Agency’s approach to undertaking institutional audits 
draws upon the practices and process standards developed 
and enhanced by its predecessor bodies. Since those bodies 
began their work, good practice in auditing (guided by 
published standards of auditing practice) and requirements 
relating to accountability and reporting have developed 
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considerably. The Agency recognizes that some of the 
process standards it has observed in the past have been 
implicit rather than explicit and that the institutional audit 
process should be underpinned by a more explicit 
statement on operational principles and process standards. 
2. In developing its operational principles and process 
standards, the Agency has taken note of the principles 
underpinning the AAIOOO series accountabrlity standard 
and the “Seven Principles of Public Life” developed by the 
Nolan Committee. 

Principles 

1. The Agency seeks to observe and promote several general 
principles within both the strategic and operational levels of 
its work. The principles are: 

Inclusiveness - taking into account the needs of all 
stakeholder groups and facilitating their participation in 
aspects of the Agency's work. 

Openness - transparency in the work and methods of the 
Agency, to build trust and confidence among stakeholders, 
and to provide information about the Agency's work to the 
wider public. 

Accountabrjiy - demonstrating that the Agency is using its 
resources to good effect and with probity; conducting its 
work with integrity and impartiality; and ensuring that 
stakeholders are able to depend on the information 
provided. 

Timeliness - the need for regular, systematic, and timely 
action in all reporting processes to support the decision- 
making of the Agency and its stakeholders. 

CoMPARABEHY - using experience drawn from within the 
Agency and other organizations as a means with which to 
inform future work. 

Relevance - ensuring that the information provided by the 
Agency is useful to, and understood by, aU stakeholders. 
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2. In due course, these principles will be used to develop 
explicit service standards for institutional audits. 

Quality Assurance Mechanisms 

1. The Agency is committed to the regular monitoring and 
evaluation of its policies, procedures, and processes, to ensure 
their ongoing credibility, and to continuously improve its 
performance in response to the results. In respect of 
institutional audit, this commitment includes providing the 
opportunity for participants in the process, including students, 
to provide structured feedback on their experiences. 

4.1.5. United States of America: CHEIA <http://www.chea.org> 

Statement on Good Practices and Shared Responsibility in the 
Conduct of Specialized and Professional Accreditation Review 

[Note: specialized accreditors are those who accredit 

programmes only and on a national rather than a regional 
basis. Some of these accreditors also operate outside the 
United States) 

Key issues addressed by the statement: 

- clear and direct communication between specialized 
accreditors and institutional leaders; 

- enhanced understanding by specialized accreditors of 
the larger context of institutional needs and direction; 

- enhanced understanding by institutional leaders of the 
perspective and needs of specialized accreditors; and 

- affirmation that the relationship between resources and 
accountability is grounded in meeting accreditation 
standards. 

Full text on <http://chea.org/research/good-practices.cfm>. 
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4.2. QUALITY ASSURANCE: SELF-EVALUATION DOCUMENTS 
AND REPORTS 

4.2.1. United Kingdom: Guidance to Institutions on Producing 
a Self-Evaluation - An Example from the “Draft Handbook on 
Institutional Audit in England’' (April 2001, full text of 
Handbook available at http://www.qaa.ac.uk) 

Annex C: Guidelines for Producing Self-Evaluation 

Documents for Discipline Audit Trails 

11. 1. After the audit team has confirmed the discipline 
audit trails to be pursued during the audit visit, the 
institution is asked to provide a self-evaluation for each 
discipline to be trailed (the discipline SED). A recent 
internal review report (or similar) on the discipline, 
accompanied by the relevant programme specifications, 
may be provided instead of a discipline SED, providing that 
it covers the matters listed below, in Paragraph 2, or is 
supplemented with a note covering these matters. 

o 

Length, Style, Content, and Structure 

1. If the institution wishes to prepare a discipline SED 
specifically for the purposes of the audit trail, it is suggested 
that the document [be] about 3,000 words in length and 
cover the following: 

Educational Aims of the Provision - a statement of the 
overall aims of the programme or cluster of programmes 
covered by the discipline audit trail; 

Learning Outcomes - evaluation of the appropriateness, to 
the educational aims, of the intended learning outcomes of 
the programme or of each of the clusters of programmes, 
making reference to internal and external reference points 
such as Subject benchmark statements and the FHEQ 
(Framework for Higher Education Qualifications - England, 
Wales, and Northern Ireland); 

Curricula AND Assessment - evaluation of the ways in which 
programme content and methods of assessment support 
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achievement of the intended learning outcomes of the 
programme(s); how curricula and assessment together 
determine the academic level of the award(s) to which the 
programme(s) lead; the extent to which students achieve the 
programme aims and intended learning outcomes. 

Quality of Learning Opportunitie^s, which can be further 
divided into: 

Teaching and Learning - evaluation of the effectiveness of 
the teaching and learning strategies employed by the 
programme(s) for providing students with good learning 
opportunities to support achievement of the intended 
learning outcomes and academic standards; 

Student Admission and Progression - evaluation of the 
ways in which student progression through the 
programme(s) is supported and monitored, from intake 
to completion; 

Learning Resources - evaluation of effectiveness of the 
deployment of the resources, human and material, that 
support the learning of students, and of the effectiveness 
of their linkage to the intended learning outcomes of the 
programme(s). 

Maintenance and Enhancement of Standards and Quality - 
evaluation of the effectiveness of procedures for 
maintaining and enhancing the quality of provision and 
the security of academic standards in respect of the 
programme(s). 

Annex D - Programme specifications for each programme 

COVERED BY THE TRAIL 

1. The emphasis in the discipline SED [self-education 
document] should be on evaluation of student achievement 
of the appropriate academic standards, and of the learning 
opportunities offered to students to support their 
achievements. Description of the programme(s) should be 
the minimum necessary to enable the audit team to 
understand the background of the self-evaluation. The 
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programme specifications may be able to cover most or all 
of the descriptive materiad that is needed. 

2. Discipline SEDs should: 

- be balanced and relevant; 

- be concise and accessible to the audit team; 

- be appropriate^ balanced between analysis and description. 

3. Institutions will note the advantages of ensuring that their 
intemad processes are capable of generating reports that 
can serve, with minor adaptation, as discipline SEDs. 

Submission 

1. The institution is required to submit discipline SEDs to the 
Agency seven weeks before the audit visit. 

CONFIDENTIAIirY 

2. Discipline SEDs remain confidential to the Agency and the 
audit team; however, when appropriate they will be made 
available to specialist advisers who are asked to provide 
advice to the team, and to subject specialist reviewers 
undertaking reviews in the institution. It is likely that the 
audit report will refer to and include quotations from the 
discipline SEDs. The institution is strongly encouraged to 
involve students in the preparation of discipline SEDs and 
to make the completed documents available to them. 

4.2.2. Germany: FH Conference I FH Council: Quality System 
<http://www.fhr.ac.at/english> 

(In this example, reference is made to legislation, the FH 
Studies Act, so as to show how the agency is addressing the 
requirements of the legislation.) 
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Form and Content of the Self-Evaluation Report and the 
Peer Report 

1 . Mission and Goals 

Main mission derived from the FH Studies Act and the 
application for a FH study programme and the goals derived 
there from: 

- Normative concept with the “vision” of the study 
programme; 

- Strategic concept with the short, medium, and long-term 
strategies and the definition of the core competencies; 

- Operative concept with the implementation of the 
measures derived from points 1 and 2 as well as 
communication of the mission. 

Section 3, Paragraph 1: “Fachhochschule study programmes 
are programmes at higher education level which serve to 
impart scientifically sound professional education”. A 
prerequisite for the recognition as a FH study programme 
pursuant to Section 12, Paragraph 2, Sub-paragraph 1 is that 
the “goals and the guiding principles for the design of 
Fachhochschule study programmes (Section 3) are met. 

2. Pedagogical Dm Acnc Concept 

- pedagogical didactic basic concept; 

- description of forms of teaching and their 
implementation; 

- balance between types of courses; 

- measures for students from second chance education. 

Section 3, Paragraph 2, Sub-paragraph 1: “Fachhochschule 
study programmes have to take into account the variety of 
scientific study methods”. Section 3, Paragraph 2, Sub- 
paragraph 8: “The didactic concept of the courses has to 
correspond to their purpose and the educational level of the 
students”. 




PROCESSES, APPROACHES, AND NETWORKING 147 



Target group specific FH programmes (Compare information 
for applicants as amended). The use of distance study 
elements is obligatory for this special form of study 
programme: the presentation of a matching scientific and 
didactic concept as weU as an appropriate quality assurance 
system is essential. 

Section 4, Paragraph 2, Second sentence: “If the scientific 
concept of an FH study programme is based on professional 
experience, the access to this FH study programme may be 
restricted to an appropriate target group”. If the access is 
restricted to a target group, “then the duration of studies is six 
terms; the FH study programmes have to be established as 
distance studies” (Section 3, Paragraph 2, Sub-paragraph 2). 

3. Curriculum and Modification Procedure 

- Teaching goals, teaching content, didactic structure per 
department and course; 

- (Special impact of the required professional experience 
on the curriculum in target group-specific study 
programmes); 

- How is the topicality examined, which body in the study 
programme decides on proposed modifications to the 
curriculum; 

- Coordination of courses and teaching contents. 

4. Evaluation OF THE Teaching 

- To what extent will the general objectives (Point 1) and 
the detciiled teaching goals (Point 3) be achieved? 

- Self-evaluation by the teachers; 

- Course analysis by the students and the study 
programme management (collegium); 

- Consequences, measures, and their implementation; 

- Results of current improvement processes; 

- Evaluation of examinations. 
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5. Admission Procedure 

- Application of the admission criteria according to the 
application for an FH study programme; 

- Body of experts (team or individual person); 

- Permeability of the education system; 

- Admission statistics of the last years; 

- Recognition of proven knowledge. 

6. Analysis of Course of Studies 

- Number of study applicants and admitted students, 
drop-out rate, marks, etc.; 

- Measures in case of [for instance], a high drop-out rate, 
unsatisfactory analysis of graduates, unsatisfactory 
average mark; 

- Measures to ensure that the prescribed duration of 
studies also corresponds to the actual duration. 
Analogous to the statistical analysis [for] the FH Council 
in accordance with the “Regulation on Making Available 
Information on the Studies as Amended”. 

7. Resources /Equipment /Finances 

- Key data on facilities and physical assets according to 
the application for an FH study programme; 

- Library [and] teaching aids... including EDP equipment, 
student guidance and infrastructure (guarantee in case 
of study programmes for working people?); 

- Financial expenditure for teaching, administration, 
equipment; 

- Costs per student. 

8. Teaching Staff 

- Selection of teachers; 

- Compliance with minimum requirements according to 
the FH Studies Act; 

- Qualification profile of teachers; 
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- Measures for personnel development and further education 
of teachers and employees; 

- Number of full-time teachers. 

9 . Practical Training and Diploma Theses 

- Evidence of practical training places relevant to study 
programme; 

- Placing of trainees with regard to industrial law and 
social legislation; 

- Documentation of practical guidance; 

- Documentation and guidance of thesis and details of the 
thesis profile. 

1 0. Analysis of Graduates 

- Integration and success on labour market; 

- Profession-related activity; 

- Duration of job search; 

- Salary; 

- Doctoral programme. 

J J . R(esearch) & D(evelopment) AcnvYHES 

- Publications; 

- R&D projects (number and project volume from projects 
funded by third parties and project expenditure); 

- Services (training and consultation). 

- Conferences and events. 

1 2 . Activities at Home and Abroad 

- Co-operation partners at home and abroad (number and 
name of institutions); 

- Student mobility; 

- FH lecturers mobility; 

- International and national meetings, conferences, and 
events (both participation and organization); 
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- International and national projects and co-operation 
[e.g., development of curriculum, ECTS, intensive 
programmes, etc.); 

- Public relations (information and guidance by FH study 
programme). 

13. Organization 

- Structure and sequence (study programme, maintainer); 

- Competencies and structure of responsibilities; 

- Autonomy of the study programme according to the FH 
Studies Act. 

14. Q(uALnr) M(anagement) System 

- Characteristics of the quality control policy; 

- Presentation of the applied quality management system 
(system description, control systems, responsibilities). 

FH Studies Act Commentary 

The achievement of the goals and the guarantee of the 
principles of Fachhochschule study programmes require 
internal measures for quality assurance which should be 
integrated into an extensive quality management system and 
linked with each other. 

15. Self-Evaluation Report 

Point 15 of the peer report should give its view on the quality of 
the self-evaluation report and assess the report. 

4.3. THE SELECTION, APPOINTMENT, AND TRAINING OF 
PEERS: SOME EXAMPLES OF CURRENT PRACTICE 

4.3.1. Austria: FH Conference /FH Council 
<http://fhr.ac.at/english> 

Composition of the Peer Group 

1. The peer group consists of four peers and an assistant and 
is made up as follows: 
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- a representative from the pool of experts of the FH 
Council; 

- a representative of the Committee for quality matters of 
the FH Conference; 

- an expert of the professional field; 

- a representative of a subject-related educational 
institution from abroad or with relevant experience 
abroad; 

- an assistant of the administration/management of an 
Austrian FH study programme (with right of proposal 
but not of voting). 

2. The guidelines for the nomination of the peers. 

- The peer of the FH Council should have a subject- 
relevant or pedagogical didactic qualification. 

- The FH study programme to be evaluated has a single, 
justified veto right concerning the peer from the Quality 
matters of the FH Conference. (An example would be the 
geographic proximity of a peer from a subject-related FH 
study programme.) 

- In order to guarantee the impartiality of the external 
evaluation, close relationships of peers with the FH 
programme to be evaluated have to be avoided. Explicitly 
excluded are teachers of the FH study programmes, 
members of the development team, and persons in 
contractual dependence. 

Workshop for Peer Groups 

- Workshops are designed and commissioned for all 
members of peer groups. 

- Two weeks before the workshop th^r receive comprehensive 
written documentation on the general statistical analyses 
and the regulatory and control mechanisms of the 
Austrian FH sector, the QS system FH Conference/FH 
Council, and the legal foundations. 

- The workshop is carried out in discursive form and 
includes the experience of senior peers. 



152 



C. CAMPBELL and C. ROZSNYAI 



4.3.2. Denmark: Evaluation Group (EVA) <http:i lwww.eva.dk> 

For each evaluation, EVA appoints an evaluation group. The 
group is made up of individuals with special expertise in the 
field concerned. The quality and integrity of the members of 
the evaluation group are crucial. All members must be 
independent of the programmes/institutions being evaluated, 
As a general rule, EVA tries to recruit at least one Nordic 
member for each evaluation. 

4.3.3. The United Kingdom: The Quality Assurance Agency (QAA) 
< http: / / WWW. qaa. ac. uk> 

All auditors, audit secretaries, and speciadist advisors are 
selected by the Agency on the basis of published selection 
criteria and generally from nominations made by institutions. 
All are provided with induction and training to ensure that 
they are familiar with the aims, objectives, and procedures of 
the audit process and with their own roles within it. The 
qualities required in auditors, etc., are outlined in the QAA 
Draft Handbook for institutional audit, and every attempt is 
made to ensure that the cohorts of auditors and specialist 
advisors reflect appropriate sectoral, discipline, geographical, 
gender, and ethnic balance. Training for auditors, audit 
secretaries, and specialist advisers is undertaken by the 
Agency in collaboration with appropriate training providers. 
The purpose of the training is to ensure that all: 

- understand the aims and objectives of the audit process; 

- are acquainted with the procedures involved; 

- understand their own roles and tasks, the importance of 
team coherence, the Agency’s expectations of them, and 
the rules of conduct governing the process; 

- have an opportunity to explore and practice the 
techniques of data assimilation and analysis, the 
development of programmes for visits, the construction 
and testing of hypotheses, the forming of judgments and 
statements of confidence, and the preparation of reports. 
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4.4. INTERNATIONAL BENCHMARKING AND EVALUATION: 
THE DANISH EVALUATION AGENCY (EVA) - AN 
INTERNATIONAL COMPARATIVE EVALUATION OF 
PROGRAMMES IN AGRICULTURAL SCIENCE 

Introduction 

This example presents an innovative project which could prove 
useful, particularly for co-operation among small higher 
education systems and especially when only one or two 
national institutions offer programmes in a paiUcuIar subject 
area. 

Although there have been some pilot projects on evaluating 
programmes according to a shared methodological framework, 
there have been few attempts to set up transnational 
evaluations with a comparative perspective. 

The Danish Evaluation Institute (EVA) decided in its action 
plan for 2001 to initiate an international comparative 
evaluation within the field of higher education. The scope of 
the evaluation is a pilot project involving BSc programmes in 
Agricultural Science in institutions in four countries - the 
Royal Veterinary and Agricultural University of Denmark, 
Waginingen University in the Netherlands, the University of 
Hohenheim in Germany, and University College, Dublin, 
Ireland. 

In developing the methodological framework for the pilot 
project, EVA has drawn on the lessons learned from the 
relatively few international evaluations which have been 
conducted during the past ten years and the substantial 
experiences gained by the cycle of evaluations of higher 
education programmes conducted by the predecessor of EVA, 
the Danish Center for Quality Assurance and Evaluation of 
Higher Education. The criteria for selecting the institutions to 
participate were as follows: 

- The institutions should have a record of commitment to 
the internationalization of higher education. 

- The institution should be expected to be motivated to 
participate in the evaluation and accordingly to allocate 
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the necessary time and human resources involved 
primarily in the self-evaluation process and in the follow 
up on evaluation. 

- The institutions should be able to appoint representatives 
for all relevant groups of stakeholders who are able and 
willing to communicate in English. 

Purpose of the Evaluation 

The purpose of the evaluation is two-fold. The project will 
partly support the development of a common framework for 
international comparative evaluations and partly provide the 
participating institutions with substantial reporting on the 
quality of their teaching and learning in the field of agricultural 
science. Following this [plan], the specific objectives of the 
evaluation are to: 

- develop and test a common methodological framework and 
common quality criteria for comparative international 
evaluations of programmes within higher education; 

- stimulate discussions on what constitutes good quality 
within higher education across countries; 

- establish mechanisms for continuous quality improvement 
and co-operation between participating institutions. 

Areas of Focus 

In its evaluations, EVA usually covers a broad range of aspects 
related to the programmes being evaluated. As a pilot project 
with a strong methodological focus, the international 
evaluation will only include a few aspects. This choice reflects 
conclusions drawn from earlier international evaluations 
stressing the importance of limiting focus when conducting 
evaluations across different educational cultures. There will be 
three areas of focus: 

1. Core Competencies 

The assessment will include the characteristics of methods 
related to competencies as well as to competencies specifically 
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related to agricultural science gained through the completion 
of a degree in agricultural science. 

2. Learning and Quality Assurance Mechanisms 
The assessment will include three elements: 

- the existence of mechanisms for quality assurance and 
monitoring of on-going activities; 

- the existence of mechanisms and systems for 
documentation and dissemination of experiences and 
lessons learned; 

- the capacity to transform gained experiences and lessons 
learned into changed practices and strategies. 

3. Internationalization 
The assessment will include: 

- the degree of internationalization in terms of the content 
of the programme [for] international co-operation, among 
other things, [and] the level and scope of international 
exchange of students and staff; 

- existing procedures for exchanging best practices and 
benchmarks of quality. 

4. Organization 

A team of evaluation officers from EVA will be responsible for 
the practical and methodological planning and implementation 
of the evaluation, while a panel of international experts (from 
Germany, the Netherlands, the United Kingdom, Norway, as 
well as Denmark) will be responsible for the professional 
quality of the evaluation. The specific tasks of the experts 
include mainly involvement in the formulation of common 
quality criteria and responsibility for drawing conclusions and 
recommendations based upon the documentation from the 
institutions involved in the evaluation. The composition of the 
panel will thus involve persons with strong methodological 
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skills as weU as persons with expertise within the field of 
agricultural science. 

5. Evaluation Metthod 

This includes a number of different elements generally used by 
EVA. 

L Preliminary Study 

EVA conducts a preliminary study (desk study) to collect 
and review existing materials relating to the field of 
education (agricultural science) in the countries involved. 
EVA also collects and studies the findings and 
methodological considerations of other international 
evaluations with a similar focus on comparative analyses. 
The preliminary study concludes with the formulation of the 
terms of reference and the appointment of the panel of 
experts. 

it Self Assessment 

The participating institutions (programmes) conduct a self- 
assessment analyzing and assessing their ovm strengths 
and weaknesses in relation to the three selected focus areas 
(core competencies, learning and quality assurance 
mechanisms, and internationalization). 

Hi Visits 

The international evaluation panel will visit the involved 
institutions. The visit will be planned in co-operation with 
the institutions and will, togetoer with the self-assessment 
reports, constitute a substantial part of the background for 
the findings and conclusions of the evaluation. 

iv. Reporting 

The analysis, assessment, and recommendations of the 
evaluation will be documented in a report. A draft report 
will be sent to the institutions involved prior to completion 
of the final report. The final report will be published. 
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V. Follow-Up 

The follow-up process will be deteirnined later together with 
the institutions concerned but will possibly consist of an 
international conference focusing on the methodological as 
well as the programme related outcome of the evaluation. 
There will be no standard setting, and the assessment will 
not result in any ranking of the programmes or the 
institutions. Rather, the assessment will focus on the 
strengths and weaknesses of the priorities of the individual 
programmes in relation to the areas of focus in the 
evaluation. The intention of using the criteria in this way is 
to encourage and stimulate the development of the 
programmes involved in the evaluation. The EVA stresses 
that the methodological approach of the evaluation and 
application of common quality criteria must not be 
interpreted as an attempt to set up and to use an 
accreditation procedure. Although accreditation requires 
the use of common quality criteria, application of criteria 
does not, per se, imply accreditation. 

Further details, including the final reports, in English, on 
the evaluations will be available at a later date from EVA; 
see <http://www.eva.dk>. 

4.5. APPROACH TO QUALITY ASSURANCE: THE EXAMPLE 
OF AN AGENCY IN A CENTRAL AND EASTERN EUROPEAN 
COUNTRY - THE HUNGARIAN ACCREDITATION 
COMMITTEE (HAC) 

The Hungarian Accreditation Committee (HAC) is preparing for its 
second eight-year cycle of institutional accreditation. In 
preparation for developing its new guidebook and to set down 
its general aims for the coming years, it has produced a strategic 
plan <http:/ /www.mab.hu/en^h/a_regulations.html> (summaiy). 

/. The Main Issues 

The mission of the HAC is to contribute to the improvement of 
the quality of life of Hungarian society through improving the 
quality of higher education. 



158 



C. CAMPBELL and C. ROZSNYAI 



The general aim of the HAC is the protection of the quality 
of Hungarian higher education. Quality is interpreted as 
fitness for purpose, which includes the capacity to meet 
stakeholder expectations. 

The HAC has the following powers: 

a. It makes recommendations in the field of educational 
policy based on discussions pertaining to general issues 
of the quaUty of Hungarian higher education. 

b. It issues opinions to the Minister, to the Higher 
Education and Research Council, and to the higher 
education institutions on matters determined by the 
Higher Education Act (accreditation tasks). 

c. It advises the institutions and programmes to assist 
them in their qualify enhancement (as part of the 
accreditation processes). [This function has not been 
emphasized so far.) 

d. It issues decisions on launching and operating doctoral 
schools and on the disciplines in which an institution 
may offer post-graduate and doctoral training. 

n . International Tasks 

The HAC will initiate mutual recognition with foreign 
accreditation and/or quality assurance agencies. The HAC 
initiates the harmonization of the work of CEE quality 
assurance organizations. 

The HAC maintains and, as far as possible, develops its 
research and information activities concerning foreign quality 
assurance activities and organizations. The results of these 
activities must be utilized in updating quality requirements 
and procedures, in producing various guidebooks and 
documents, and in improving its operation. 

iff. Institutional Accreditation 

Institutional accreditation will have four essential, interrelated 
components in the future: 

- public presentation of the general and specific data 
related to the quality of institutions (made by the 




.1 53 



PROCESSES. APPROACHES. AND NETWORKING 1 59 



institutions themselves, on their homepages, updated 
yearly) [New feature]. 

- self-evaluation report by the institution based on its 
SWOT and constraints analyses, with special regard to 
the process and the outcomes of teaching and learning 
[Includes new features). 

- site visit and formative report by ad hoc visiting 
committees. 

- user surveys by the institutions to determine user 
satisfaction [New feature). 

The HAG is deterrnined to strearnline the process of 
institutional accreditation. A new Accreditation Guidebook is 
being written. Fewer data will be asked for in the self- 
evaluation report, while the analytical, evaluative character of 
it will be strengthened (SWOT analysis). Institutions will be 
asked to name and evaluate their best and worst programmes 
and fields of operation in their self-evaluation. Institutions are 
asked to put quality-related data and information on their 
Website. The aim is to serve not only accreditation purposes 
but to provide information to students, other stakeholders, 
other (competing) institutions, and the general public. 
Moreover, easily accessible public data will improve the 
transparency and accountability of the system. 

The HAG will evaluate the mission and goals of the 
institutions and also other factors determining the quality of 
the institution as a whole {e.g., management, internal quality 
assessment system). Moreover, the HAG will lay more 
emphasis than before on evaluating the process and outcomes 
of teaching and learning. 

As for programme accreditation performed within the 
framework of institutional accreditation*, the HAG will not 



In the first cycle of institutional accreditation, all running programmes of the 
institution were evaluated in detail. By "programme accreditation performed within the 
framework cf institutional accreditation", this type of activity is meant. By programme 
accreditation in itself we mean a separate activity, the accreditation of programmes to be 
established or launched (this is called “preliminaiy accreditation” according to the 
terminology of the new government decree on the HAC). 
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examine aU programmes as was the case in the first cycle. The 
following programmes wiU be evcduated in detail: 

- programmes judged in the first cycle to require 
monitoring; 

- programmes launched after the first cycle institutioncd 
evcduation report; 

- selected programmes from among the best and worst 
named in the SER; 

- programmes selected at random. 

The institutional accreditation report will contain: 

- an evaluation of the institution; 

- discussion of the interned quality assessment system; 

- detailed evaluation of selected programmes; 

- discussion of data and information coming from users. 

4. Programme Accreditation 

Programme accreditation continues according to earlier 
practice. An important new feature wiU be that the HAC will 
launch pilot study programme evaluations of programmes in 
the same disciplines at various institutions, probably in 2003. 
The procedure and the standards of excellence must be 
worked out by end of June 2003. 

After the ftill accreditation of doctoral schools is completed, 
a comprehensive report will be produced by end of June 2002. 

5. Tasks Related to the Operation and Organization of the HAC 
The most important tasks of the HAC for improving its 
operation are the following: 

- creating an internal system of quality assurance (by the 
end of June 2003); 

- yearly review of its operation; 

- detailed analytical self-evaluation every three years, 
discussed with stakeholders (due in 2003); 

- review of the current subcommittee structure (due in 
2003). 
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6. Methodology 

The activities of the HAC will continue to have a double focus, 
in the future, from the point of view of methodology. On the 
one hand, the HAC performs accreditation, Le., it defines the 
quality of institutions, faculties, and programmes in relation to 
predetermined threshold quality requirements, Le., minimum 
standards (judgments: “yes”, “conditional yes”, and “no”). On 
the other hand, above the threshold level, the HAC gives a 
detailed assessment and recommendations as to the 
enhancement of the quality of the given institution, faculty, or 
programme. That means that beyond the so-called “control” 
function (giving opinions to the Minister), the HAC also 
performs an advisory function to promote quality enhancement. 
It is the definite intention of the HAC that in the future the 
latter function be more emphatic than the former. 

An important change in this respect is that in the second 
cycle of institutional accreditation the HAC wiU no longer grade 
degree programmes. (In the first cycle we used the grades 
“Excellent”, “Strong”, “Adequate”, “Inadequate”.) 

7. Publicity 

The activities of the HAC should be publicized more widely and 
effectively in the future. The aim of the corrunimication strategy 
of the HAC is to improve the image of the organization and to 
deliver the mission, the aims, and the outcomes of the HAC’s 
operation to various target groups (higher education 
institutions, students, the Ministry of Education, and the 
general public). 

4.6. QUALITY ASSURANCE FROM THE PERSPECTIVE OF A 
UMVERSHY NETWORK ASSOCIATION: A POLICY STATEMENT 
OF THE EUROPEAN UNIVERSITY ASSOCIATION (EUA) 

1 . Background 

This EUA policy paper on quality assurance arises from five 
key developments that have taken place in Europe over the 
past few years: 
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The Magna Chorta Universitatum (1988) which upholds 
university autonomy, must be the precondition for fostering 
the adaptability of universities to the ever-changing 
requirements of today’s society. 

The meeting of Ministers at the Sorbonne's 800th 
anniversaiy (1998) referred to the central role of higher 
education in the development of Europe through the creation 
of a European Higher Education Area. 

[Through] the Bologna Declaration (1999), ...the twenty- 
nine signatory states agreed to act in concert to increase the 
competitiveness of Europe through a range of measures aimed 
at creating a European Higher Education Area. These include 
the adoption of a system of easi^ readable and comparable 
degrees and a system of credits. iTie objectives of such tools 
are to promote mobility, European co-operation in quality 
assurance, inter-institutional co-operation, and integrated 
programmes of study, training, and research. 

The Salamanca Convention (2001) of European higher 
education institutions considered quality as a fundamental 
building block of the European Higher Education Area and 
made it the underlying condition for trust, relevance of 
degrees, mobility, compatibility, and attractiveness. 

Similarly, the recent Communique of the European 
Education Ministers (Prague, 2001) regards quality as a major 
factor in determining the competitiveness and attractiveness of 
European higher education. 

In this context, the European University Association 
confirms the central role of quality in higher education and 
affirms that the evaluation of quality should: 

- be based on trust and co-operation between institutions 
and evaluation agencies; 

- take into account the goals and missions of institutions 
and programmes; 

- consider the balance between tradition and innovation, 
academic excellence and socio-economic relevance, the 
coherence of curricula, and students’ freedom of choice; 
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- examine teaching and research as well as management 
and administration; 

- include responsiveness to the needs of students and the 
provision of non-educational services. 

Quality assurance refers to a set of procedures adopted by 
higher education institutions, national education systems, and 
international agencies through which quality is maintained 
and enhanced. 

Quality assurance is effective when it refers to the very core 
of the higher education activity and when its results are made 
public. 

Quality assurance impUes academic autonomy and is 
closely dependent on academic management that is based on 
the principles of efficacy, academic and scientific performance, 
as well as competitiveness. 

Quality assurance can succeed only if it becomes inherent to 
the institutional culture. Such a culture generates the necessary 
motivation and ensures competence in implementing quality 
assurance mechanisms. 

2. Benchmarking Quality Management 

Various higher education systems have developed policies 
regarding quality criteria, quality assessment, quality 
assurance, and quality management. The quality of higher 
education can be defined in various ways as: excellence, “zero 
defects”, “goal adequacy”, capacity for ongoing improvement, 
minimum standard, marketability, or competitiveness. Every 
approach has its contextual justification. 

Quality starts by ensuring mirilmum standards. It extends 
to the capacity of ongoing improvement and includes a 
competitive dimension at the national and at international 
levels. 

The European University Association considers that it is 
important to identify common benchmarks for quality 
management and assurance to contribute to the creation of 
the European Higher Education Area. These quality 
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benchmarks must focus on the multiple dimensions of 
academic activities, as follows: 

a. academic autonomy as an instrument for improved 
performance and competitiveness; 

b. explicit institutional mission and objectives of 
institutions and programmes; 

c. transparent and non-discrirninatoiy access and 
recruitment policies, the possibility of a second chance, 
and fair appeals policies; 

d. curricular quality; 

e. academic staff quality; 

f permanent feedback from the students and 

responsiveness to their suggestions, proposals, and 
critique; 

g. flexible organization allowing credit transfer, 
interdisciplinarity, and studying within the framework of 
various programmes or institutions; 

h. quality of infrastructure and availability of adequate 
equipment; 

i. resource allocation with the capacity of obtaining extra- 
budgetary resources, motivating academic staff, and 
investing in buildings and equipment; 

j. accountability with regard to the use of human and 
material resources and systematic auditing; 

k. feedback from stakeholders and the possibility of 
adapting degree programmes to labour-market needs; 

l. international scientific competitiveness; 

m. internal quality assurance mechanisms; 

n. contribution to public debate and democracy; 

o. innovation potential in technical, scientific, cultural, and 
artistic fields. 

3. Accreditation Principles 

Various countries have developed specific accreditation 
systems. The European University Association considers 
accreditation as one possible outcome of quality assurance 
and defines it as a formal recognition of the fulfillment of 
rnmimum, publicly stated standards referring to the quality of 
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a programme or an institution. Accreditation is the adequate 
mechanism for assuring minimum standards of education 
and, in some cases, can be seen as the first step toward 
quality. It must be used, however, in combination with robust 
institutional quality review. 

Periodic self-assessment of each institution or programme is 
an important step in quality assurance. Self-assessment carries 
more weight; however, [only] if it is accompanied by an external 
assessment phase performed by independent assessment 
agencies. In turn, ensuring an international dimension will 
contribute to the quality of national assessments. 

Communication among national systems is still poor, and 
there is a quality information gap. It is necessaiy that different 
national systems accept a univocal significance of accreditation. 

The European University Association considers that the 
time has come to take steps towards making [the] accreditation 
standards of various European countries compatible with one 
another, through bilater^ or multilateral agreements. At 
present, however, there is no need to develop a single 
European accreditation system, but it is timely to think about 
criteria and mechanisms to validate the accreditation 
procedures applied in Europe. 

Specifically, the European University Association regards 
the following principles as central to accreditation procedures. 
They must be: 

a. geared at quality enhancement which means that the 
process will focus on the internal quality control 
mechanisms in the institution and insure that these are 
used for strategic planning; 

b. preserve institutional diversity and autonomy as well as 
foster innovation by evaluating institutions against their 
missions and strategic plans; 

c. assure public accountability by (i) including stakeholders 
in the process, (ii) communicating the results to the 
public, and (Hi) being independent of governments, 
interest groups, and higher education institutions; 
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d. consist of a self-evaluation and an external assessment 
with a clear emphasis on self-evaluation as a formative 
step in institutional planning; 

e. have guidelines that are transparent to the higher 
education institutions and the public; 

f. set up a procedure that makes clear distinctions between 
conditions for accreditation and recommendations for 
improvement; 

g. have a specified and fair appeals procedure; 

h. be re-assessed on a cyclical basis in terms of the adequacy 
of an agency’s resources and its impact on institutions. 

The European University Association encourages institutions 
to ensure the internal review of their programmes and 
supports initiatives in Europe to promote defined and 
appropriate mechanisms for the accreditation of institutions. 

4. European and International Co-operation 
In the context of globalization and internationalization, quality 
assessment implies, more than ever, comparing approaches 
and results, as well as learning from good practice. It is 
necessary and beneficial to extend international co-operation 
among institutions in view of implementing quality assessment 
and assurance mechanisms, improving the assessment of 
academic programraes, sharing assessment methods, and 
exchanging experience. 

The European University Association encourages the 
networking of institutions in matters of comparing 
organizations and results, ensuring co-operation in designing 
and improving quality assessment methods, comparing and 
developing quality assessment systems, making public 
examples of good practice, and sharing experience [e.g., on the 
introduction of new degrees or ECTS) - these are effective 
means to consolidate the quality of higher education 
programmes. To this end, the European University Association 
will co-operate closely with ENQA (European Network for 
Quality Assurance), ESIB, OECD, UNESCO-CEPES, and other 
international organizations and institutions concerned with the 
quality of higher education. 
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The European University Association supports the 
development of a database regarding higher education 
systems, dissemination of information on innovation, and 
elaboration of comparative studies of systems and institutions. 

5. Conclusion 

In the present context of changes brought about by 
globalization and the internationalization of the academic 
sector, the European University Association and its members 
support steps taken by institutions, in partnerships with 
governments, towards: 

a) operating changes towards the expansion of the 
European dimension of higher education and the 
creation of the European Higher Education Area; 

b) curricular reform; 

c) improving academic management, including the capacity 
for internal quality management; 

d) developing universities as teaching, learning, research, 
and service-providing units; 

e) expanding and consolidating scientific research in 
universities; 

J) adopting compatible mechanisms for quality assessment; 
g) achieving convergent education systems by rendering 
them comparable and compatible, based on common 
denominators with a European dimension. 

(Approved by the EUA Council, Dubrovnik, 27 September 2001 
<http://www.unige.ch/eua>) 

4.7. ACCREDITATION AND RECOGNITION: SOME 
INTERNATIONAL INITIATTVES BY THE ENGINEERING 
PROFESSION 

4.7.1. The European Federation of Nationol Engineering 
Associations (FEANIj <http://www.feani.org> 

The European Federation of National Engineering Associations 
was established in the early 1950s and today brings together 
more than eighty national engineering associations from 
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twenty- seven European countries. Through these associations, 
FEANI represents the interests of approximately two million 
engineers in Europe. 

The objectives of FEANI are 

- to affirm the professional identity of the engineers of 
Europe; 

• by ensuring that professional qualifications of engineers 
of the member countries atre acknowledged in Europe 
and worldwide; 

• by asserting the status and responsibility of engineers 
in society; 

by safegUctrding and promoting the professional interest 
of engineers and by facilitating their fi*ee movement 
within Europe and worldwide; 

- to strive for a single voice for the engineering profession 
of Europe whilst acknowledging its diversity; 

• by developing a working co-operation with other 
international organizations concerned with engineering 
matters; 

in representing the engineers of Europe in international 
organizations aind other decision-making bodies. 

The Federation established the designation, Euring, to 
encourage the continuous improvement of the quality of 
engineers by setting, monitoring, and reviewing standards and 
to facilitate recognition and mobility. 

FEANI is different from the Washington Accord (see below) 
in that not all of its member organizations have responsibility 
for national accreditation or quality assurance of engineering 
programmes. 

4.7.2. The Washington Accord <http:/ /unvw.uxishmgtonaccorcLoig> 

The Washington Accord is a multinational agreement signed in 
1989 which: 
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- recognizes the substantial equivalency of accreditation 
systems of organizations holding signatory status and the 
engineering education of programmes accredited by them; 

- establishes that graduates of programmes accredited by 
the accreditation organizations of each member nation 
are prepared to practice engineering at the entry level. 

The member nations are: Australia, Canada, Hong Kong, 
Ireland, New Zealand, South Africa, the United Kingdom, the 
United States, and Japan (provisional status). The terms of the 
Accord state that the accreditation system, for which each 
signatory is responsible, shall be subject to a comprehensive 
review and report by representatives of the other signatories, at 
intervals of not more tiian six years. The review involves site 
visits. It is widely held as an example of good practice in the 
mutual recognition of agencies, and the Accord is now 
developing a number of quality assurance instruments such as 
draft principles for the signatories when they are operating 
outside their home territories. 

To date, European attempts to become signatories to the 
Accord have not been successful perhaps in part because there 
are no real European counterparts which meet the criteria to 
becoming signatories other than in the United Kingdom and 
Ireland. 

4.8. WHAT DO QUALITY ASSURANCE NETWORKS DO? 

THE EUROPEAN NETWORK FOR QUALITY ASSURANCE 
IN HIGHER EDUCATION (ENQHA) 

The European Network for Quality Assurance in Higfier 
Education (ENQA) was officially established in March 2000 to 
promote European Co-operation in the field of quality 
assessment among all the different actors operating in this 
arena. The idea for ENQA originated from a European PUot 
Project for Evaluating Quality in Higher Education that clearly 
demonstrated the importance of sharing Information and 
developing experience in quality assurance. This concept was 
officially given momentum by the European Council 
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Recommendation of September 1998 (98/561 /EC) on 

European co-operation in quality assurance in higher 
education as well as by the Bologna Declaration. 

The basic task of ENQA is to share experiences on a 
European level about good practices and methods of 
evaluation. This task is important as information about good 
practices and new methods and innovations should be 
available to everyone. The Network tries to disseminate 
information as much as it can through free reports and 
publications, newsletters, and the ENQA Website 
(<http://www.enqa.net>) which is updated on a regular basis. 
ENQA has, so far, published four reports. All the publications 
are available on the Website as well as in paper copies. 

ENQA is also a European education network funded by the 
European Commission and helps to bring evaluation expertise 
to rninisterial staff and to the evaluation experts of evaluation 
agencies. 

ENQA arranges workshops of its own and participates in 
others to promote contacts between European experts. The 
members of the Network Steering Group travel frequently to 
various European and international conferences as guest 
speakers. 

The membership of ENQA is open to quality assurance 
agencies, public authorities, and quality associations that work 
in one or more subject fields and that are external to higher 
education institutions. The Secretariat is often contacted by 
organizations willing to join the Network. While the organization 
welcomes new applicants, the most important issue is indeed 
transparency and delivering information. The network has 
detailed membership criteria (see its Website) which include 
evidence of the independence of the member institutions. 

The annual General Assembly is the decision-making body 
for the network and includes representatives of all the 
European Ministries of Education and member organizations. 
The Network has a ten-member Steering Group that prepares 
issues for the General Assembly and plans current and future 
activities for the Network. 
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PROCESSES, APPROACHES, AND NETWORKING 1 7 1 



The European Commission pays a contribution annually to 
the activities of ENQA, but apart from that, the member 
organizations pay an annual membership fee. 

The priority themes for ENQA for 2001-2002 are as follows: 

- the quality assurance and assessment of new forms of 
delivery; 

- the outcomes and follow-up of external assessment; 

- the use of outcomes of assessment by other stakeholders; 

- mutual recognition of the work of other quality 
assessment agencies; 

- the European dimension in quality assurance; 

- accreditation, European standards, and equivalence of 
programmes. 

The themes of upcoming ENQA seminars, which are open to 
members and non-members and provide opportunities for s taff 
from agencies and higher education institutions across Europe 
to interact and exchange views, include: 

- benchmarking between agencies; 

- staff development. 

The ENQA Secretariat comprises two persons and is 
currently located in the Finnish Higher Education Evaluation 
Council (FINHEEC), in Helsinki, Finland. Further details can 
be obtained from the Website or by contacting the Secretary, 

Kauko HamMainen 
ENQA Secretariat 
P.O. Box 1425 
Annankatu 34-36A 
FIN-00101 Helsinki 
Fax: +358-9-1607-6911 

6. Other Networks 

International Network for Quality Assurance Agencies for 
Higher Education (INQAAHE) <http://www.inqaahe.nl>. 



Annex 5 



The Design of Study Programmes: 
Establishing E^xtemal Reference Points 
for Quality and Standards 



Quality assurance and accreditation agencies (QAAAs) are not 
involved in the design and delivery of programmes of study. 
This role has been that of either rninistries and/or universities. 
However, quality assurance and accreditation agencies may 
indirectly influence the design and delivery of programmes 
through the external reference points they establish for quality 
and standards or through their evaluations and recommendations 
for action. 

Here are some examples of current work on the 
development of external reference points such as generic 
descriptors for qualifications (Bachelor’s and Master’s Degrees) 
and for subject specific descriptors. Examples are from 
regional (Germany), national (UK), and international initiatives. 
They focus on establishing descriptors for BA/MA 
qualifications, sometimes when such qualifications are new 
awards and sometimes when they are existing ones but may 
not have been weU defined in the past. Recent initiatives in 
relation to subject (or discipline) -specific descriptors are also 
evoked. This area is one the status of which is either “work-in- 
progress” or “being tested”. 

Advice is given as to where to go for further information. 
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5.1. THE DEVELOPMENT OF EXTERNAL REFERENCE 
POINTS FOR QUALITY AND STANDARDS OF STUDY 
PROGRAMMES 

5.1.1. Towards Shared Descriptors for Bachelor’s and 
Master’s Degrees: An International Approach - A Report from 
the Joint Quality Initiative Group 

A Joint Quality Initiative group (JQI), established by the 
Flemish and Dutch Ministries of Education, has been 
considering the development of descriptors for Bachelor’s and 
Master’s Degrees (BaMAa descriptors) that might be shared 
within Europe and be available for a variety of purposes 
depending on particular national, regional, or institutional 
contexts and requirements. A group with members from 
several national or regional quality assurance organizations 
(Belgium, Denmark, Germany, Ireland, the Netherlands, 
Norway, Spain, Sweden, and the United Kingdom), for both 
university and non-university higher education, has discussed 
the diverse requirements for, and characteristics of, such 
BaMa descriptors and has developed descriptors that may now 
be tested and shared. 

Several other national and regional projects have been or 
are currently working to identify the characteristics associated 
with particular higher education qualifications, and develop 
taxonomies and frameworks that clarify the relationships 
among qualifications. The work of the JQI has included 
detailed consideration of such projects and has additionally 
drawn on the outcomes of discussions in Helsinki oh common 
characteristics of Bachelor’s Degrees. The Helsinki discussions 
characterized Bachelor’s Degrees by the extent of study (years 
[3 or 4] or ECTS credits [180 or 240]). The work of the JQI 
group has been concerned with identifying the academic and 
other requirements that, as the outcomes of study, 
characterize and distinguish between Bachelor’s Degrees and 
Master’s Degrees. 

A survey was carried out amongst participants in the JQI 
project in preparation for discussions on the possible form, 
content, and application of BaMa descriptors. Responses 
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indicated a variety of needs and potential uses for such 
descriptors and ^so the importance of having a shared 
understanding of the terms used both within the descriptors 
and to describe the context(s) in which they applied. 

There was agreement that each descriptor should indicate 
an overarching summary of the outcomes of a whole 
programme of study. The descriptor should be concerned with 
the totality of study and with the student’s abilities and 
attitudes that have resulted in the award of the qualification. 
The descriptor should not be limited solely to describing the 
outcomes of units of assessment at the level of the 
qualification. Thus, the JQI group has developed a shared 
qualification descriptor not a shared level descriptor. However, 
it was noted that within some national, regional, and 
institutional contexts, there might also be a requirement for 
the local development of level descriptors. 

The JQI group discussed the merits of seeking a single 
shared descriptor for the Bachelor’s Degree and similarly, one 
for the Master’s Degree rather than seeking a process to 
demonstrate “compatibility” between descriptors developed for 
national, regional, or institutional purposes and that refiect the 
detail of local contexts. In line with the spirit of the Bologna 
Declaration, the JQI group concluded that it should seek a 
single generic descriptor for all Bachelor’s Degrees, and 
similarly, a single generic descriptor for all Master’s Degrees. 
The development of these descriptors should not hinder any 
national, regional, or local requirements for additional 
descriptors. 

There are a wide variety of programmes leading to 
Bachelor’s awards, varying in content, delivery and process, 
and nomenclature. For example, a number of countries 
differentiate between Professional Bachelor’s and Academic 
Bachelor’s awards (see ZEvA criteria, below, as an example). 
Similarly, a wide variety of programmes leading to different 
types of Master’s Degrees exists. It was agreed by the JQI 
group that the value of generic descriptors could be cross- 
referenced to more detailed programme profiles or specifications. 
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A programme specification or profile would identify the 
particular components of the programme leading to the 
qualifications; for example, it might include pre-requisites for 
entry to the programme, details of the components, their 
delivery and assessment, and any requirements relating to the 
needs of the regulated professions. Tlie form and components 
within the profile would reflect national, regional, or 
institutional contexts and be related to the needs and 
responsibilities of those awarding the qualification or 
accrediting the particular programme. 

The JQI group agreed that the shared descriptors should be 
described in a language and style that is “readable” by all who 
would have an interest in them, in particular students, their 
sponsors, employers, higher education academics and their 
managers, and society at large. 

The JQI group proposed the following as generic descriptors 
that might be useful as indicators or reference points for the 
abilities and qualities of holders of Bachelor’s and Master’s 
Degrees awarded within the European higher education area. 

Shared Descriptors for the Bachelor’s and Master’s Degrees 
Bachelor’s Degrees are awarded to students who: 

- have demonstrated knowledge and understanding in a 
field of study that builds upon and supersedes their 
general secondary education, and is typically at a level 
that, whilst supported by advanced textbooks, includes 
some aspects that will be informed by knowledge at the 
forefront of their field of study; 

- can apply their knowledge and understanding in a 
manner that indicates a professional^ approach to their 



The word, “professional”, is used in the descriptors in its broadest sense, relating to those 
attributes relevant to undertaking work or a vocation and that involves the application of 
some aspects of advanced learning. It is not used with regard to those specific 
requirements relating to regulated professions. The latter may be identified with the 
profile/specification. 
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work or vocation, and have competencies^ typically 
demonstrated through devising and sustaining arguments 
and solving problems within their fields of study; 

- have the ability to gather and interpret relevant data 
(usually within their fields of study) to inform judgments 
that include reflection on relevant social, scientific, or 
ethical issues; 

- can communicate information, ideas, problems, and 
solutions to both specialist and non-specialist audiences; 

- have developed those learning skills that are necessary 
for them to continue to undertake further study with a 
high degree of autonomy. 

Master’s Degrees^ are awarded to students who: 

- have demonstrated knowledge and understanding that is 
founded upon and extends and/or enhances that 
typically associated with Bachelor’s level and that 
provides a basis or opportunity for originality in 
developing and/or applying ideas, often within a 
research^ context; 

- can apply their knowledge and understanding and 
problem solving abilities in new or unfamiliar 
environments within broader (or multidisciplinary) 
contexts related to their fields of study; 

- have the ability to integrate knowledge, to handle 
complexity, and to formulate judgments with incomplete 
or limited information, but that include reflecting on 
social and ethical responsibiUties linked to the 
application of their knowledge and judgments; 



^ The word, “competency", is used in the descriptors in its broadest sense, allowing for a 
scale of abilities and skills. It is not used in the narrower sense identified solely on the 
basis of a yes/no assessment. 

^ Some JQI representatives suggested that MBA programmes be specifically excluded: 
others consider that MBA programmes should reflect the attributes contained within the 
shared Master descriptors. 

^ “Research” is used to cover a wide variety of activities, with the context often related to a 
field of study; the term here is used to represent a careful study or investigation based on a 
systematic understanding and critical awareness of knowledge. 
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- can communicate their conclusions, ctnd the knowledge 
and rationale underpinning these, to specialist and non- 
specialist audiences clearly and unambiguously; 

- have the learning skills to allow them to continue to 
study in a manner that may be largely self-directed or 
autonomous. 



Members of the JQI group have opened discussions about 
options of testing the BaMa descriptors in joint pilot studies 
that involve different approaches to quality assurance. Such 
studies will investigate the utility of the descriptors, and in 
particular their form, components, and levels of expectation. In 
addition to contributing to transparency concerning the nature 
of Bachelor’s cind Master’s Degree qualifications, it is 
anticipated that such transnational investigations will also 
contribute to enhancing the understcinding cind recognition of 
the various purposes and characteristics of different evaluation 
systems. 

Agencies and organizations which have participated in the 
discussions and drafting of the shared BaMa descriptors include: 



Belgium (FL) 
gAf 

Denmark 

Germany 

Ireland 

Netherlands 



Norway 

Spain 

Sweden 

United Kingdom 



:Ministiy of Education; VLIR (University 
: EVA 

: MKW Niedersachsen; ZevK Hannover 
: HETC; NQAI 

: Ministw of Education; Inspectorate; 
HBO-Raad] VSNU; 

Trailblazer Committee. 

: Network Norway Council 
: Aqenqua CaJbcdunya:, Consejo des 
Unwersidades 
: Hoskoleverket 
: QAA 



For further details, visit the JQI Website on <http://www. 
j ointquality . com> . 



5.2. THE DESIGN OF STUDY PROGRAMMES 

An example of (DRAFT) Standards for BaMa courses when 
such qualifications are a new development is taken from cin 
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accreditation and evaluation agency that participated in the 
JQI process. 

5.2.1. The Central Evaluation and Accreditation Agency, 
Hannover, Germany (ZevA) 

ZevA is a common institution of all Lower Saxon universities. It 
was established in 1995, and as of 2000, it assumed an 
accreditation as well as evaluative role. It is accredited by the 
Akkreditierungsrat and has participated in the Joint Quality 
Initiative group. 

General Standards for the Accreditation of New Degree 
Courses: Bachelor’s Degree - Master’s Degree - Continuing 
Education 

Contents 

1. Guidelines 

2. Research /Application Orientation 

3. General key Qualifications 

4. Internationalization 

5. Co-operation Agreements /Teaching Imports 

6. Student/teacher Ratios 

7. Modularization 

8. Additional general Standards of the Central Evaluation and 
Accreditation agency in Hanover 

Preamble 

With this draft, the Central Evaluation and Accreditation 
Agency in Hanover (ZevA) is attempting to develop central and 
generi standards to be applied in the accreditation of new 
degree courses. They are initially aimed at Bachelor’s and 
Master’s Degree level courses, but can also be developed to 
other courses and degrees at a later stage, for instance, for 
PhD programmes. 

The general standards of ZevA do not intend to replace any 
framework examination regulations, nor are they to be 
understood as static markers. The traditions of the individual 
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subjects and of the disciplines at the institutions of higher 
education, many of which vary greatly, make it necessary to 
deviate from certain standards here and there, for example 
concerning the question of compulsory field trips, the 
proportion of foreign languages in the degree course, study 
abroad, certain key qualifications, etc. What is important is 
that the totality of the general standards permit a general 
evaluation of the quality of teaching and studies against the 
background of the requirements laid down by the Education 
Ministers’ Conference (KMK), the Conference of Presidents and 
Rectors of Universities and Other Higher Education 
Institutions (HRK), and the Accreditation Council (AR), and 
that the trend is for at least these standards to be achieved. 
For this reason, the general standards of ZevA are also to be 
applied within the context of these general guidelines. Some 
examples of such application are the reference framework for 
Bachelor’s and Master’s Degree courses of the Accreditation 
Council and the recommendations of the KMK and HRK for 
modularization which, in their turn, are being developed 
further. 

Where degree courses deviate from these standards, the 
faculties offering them ought to give their reasons. 

Of course, the standards themselves must be continually 
re-examined, and they must be developed further. In order to 
do so, it is necessaiy to hold discussions with the institutions 
of higher education about their concepts, and also with the 
ministries of the Lander, the professions, the students, and the 
scientific and professional societies and associations. They are 
thus invited to participate actively in the discussions about the 
further development of general standards. 

In the discussion about higher education policy, the 
professional qualifications represented by the degrees, and 
therefore the relevance to practice, will be stressed as far as 
the future Bachelor’s and Master’s Degree courses are 
concerned. Thus the question of the compulsory proportion of 
practical, field trips, projects, etc., required within a Bachelor’s 
or Master’s Degree course programme must still be answered. 
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In this study, ZevA has, for the time being, omitted a 
quantitative determination, but it will discuss the matter in its 
committees. 

An additional task will consist of relating the already 
developed standards for various disciplines with those drawn 
up here. In particular, the relationship between the cross- 
section qualifications and the specialist qualifications in the 
narrower sense will play a part. Another question that is stiU 
open is whether it is not already necessary at the level of 
general standards to differentiate between the areas of Stage II 
studies, for instance between the humanities and the natural 
sciences. 

i . Guidelines 

The basic guidelines along which the degree programmes 
ought to be oriented if they are aiming at accreditation by ZevA 
are the following: 

t Graduates must meet the expectations of the higher 
education institution, the labour market, and society. 
The higher education degree awarded must be a reliable 
indicator that the relevant demands have been met. 

it The examinations must reach a level and standard 
necessary for the completion of the degree course and 
the award of the academic degree - in accordance with 
the Diploma Supplement. 

UL The curriculum must be suitable for providing the 
necessary qualifications and imparting the appropriate 
knowledge for the examinations. 

iv. The resources necessary for meeting the standards must 
be available. The organization of the course of studies, 
the teaching, and the examinations must fulfill 
appropriate conditions. 

V. The concepts on which the curriculum is based with 
regard to the qualifications to be obtained, and to the 
educational go^s deterrnining the course offered, must 
be appropriate. 
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2. Research /Appucation Orientation 

3. General Key Qualifications (of different types depending on 

THE PROFILE OF THE DEGREE COURSE) 

t knowledge of foreign languages; 

a ability to communicate, ability to work within a team, 
ability to integrate, intercultur^ competence; 

UL presentation ability, ability to head negotiations, project 
management; 

iv. ability to make use of modem information technologies, 
knowledge of electronic learning courses on the market; 

V. basic economic and legal competencies, sustainability, 
ethics of economics; 

vL mastery of research standards, research ethics for the 
BF (research oriented Bachelor’s Degree) and MF 
(research oriented Master’s Degree) courses. 

General Standards 

In the case of Bachelor’s Degree courses, greater emphasis 
should be laid, in principle, on key qualifications (as long as 
they are designed consecutively). In Bachelor’s Degree courses, 
the key qualifications [ii) and (iv) are the most important. In 
courses qualifying one for higher degrees, the key 
qualifications (Hi), (v), and (vi) are the rnost important. 
Knowledge of foreign languages is absolutely essential for all 
degree courses. 

4. Internationalization (for degree courses with foreign 

ORIENTATION) 

L study abroad (with registration at a foreign institution of 

higher education); 

- a total of two semesters as a mle; in the case of 
consecutive BF/MF; 

- in each case, either one semester or a work placement 
abroad as a mle (under the supervision of the higher 
education institutions): BA/MA 
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it Foreign LANGUAGE CLASSES 

- on the basis of appropriate standards, confident 
negotiation ability, understanding of technical/ specialist 
terminology, TOFEL, etc. 

at Modules 

- at least two modules (=between 12 and 20 hrs/wk per 
semester) must also be offered at a foreign partner 
institution of higher education. 

5. Co-operation Agreements /Teaching Imports 

The development of new Bachelor’s Degree courses and, above 
all. Master’s Degree courses, is already showing that innovative 
programmes frequently come into existence on the boundaries 
of classical subject canons. This phenomenon leads 
increasingly to so-called teaching imports from other teaching 
units of the same faculty, from the student’s own institution or 
from another higher education institution, and from research 
and practical application. This phenomenon also raises the 
question of the minimum proportion of core competencies a 
responsible faculty must have at its disposal for the degree 
course. 



Minimum 


Curricular 


Core competence 


70% 


60% 


50% 


BF 


BA 






MF 


MA 



Teaching imports must be covered by binding agreements. 
Organizational measures must be taken as a reaction to the 
increasing demands made on the coordination of teaching, 
studies, and examinations when there is a higher level of 
teaching imports. 

6. Student /Teacher Ratios 

The student/ teacher ratios must take the individual profiles of 
the degree courses into account. 
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7. Modularization 

Bachelor’s and Master’s Degree courses must be offered in a 
modularized form. 

Modularization is the concentration of subject areas (input 
orientation) on thematically and temporally rounded off, 
examinable units which are complete in themselves and which 
are provided with credit points. Modularization is thus an 
organizational principle that leads to a reorganization of the 
structure of a course of studies. In the interests of enabling 
students to complete a course, and of student international 
mobility, only in exceptional cases should modules last more 
than one semester. 

Modularization demands mutual recognition among 
institutions of higher education. Modularization must be 
linked consistently with a credit point system (in accordance 
with ECTS) which takes into account the students’ learning 
effort, and with an examination system involving continuous 
assessment. 

Modules must impart definable abilities (output 
orientation) . They can be comprised of various forms of 
teaching and learning. The descriptions of modules must 
include the effort required on the part of students and also the 
credit points to be awarded. 

The description of a module must contain the following as a 
minimum: 

t The content and qualification targets of the module; 

it Teaching forms; 

at Requirements for participation; 

iv. The usability of the module; 

V. Requirements for the awarding of credit points; 

vt Amount of work required, credit points, and grading; 

vil How frequently the modules are offered; 

viit Duration of the modules; 

VC. Teaching staff. 

Eight (+/-two) hrs/wk per semester should be taken as a 
guideline for the quantitative extent of a module. 



178 



DESIGN OF STUDY PROGRAMMES 



185 



8. ADDmoNAL General Standards of the Central Evaluation 
AND Accreditation Agency in Hanover 

Obligatory courses must be offered each semester if the degree 
course is planned to begin each semester; otherwise annually. 

Each semester, the faculty (departmental adviser) must 
publish a timetable (with details of classrooms) and a full 
description of the courses offered (module catalogue) for the 
students or provide it on the Internet. 

The admission requirements as well as the selection criteria 
for admission must be laid down in (state-approved) 
regulations. 

Modularization and the use of ECTS on the basis of 
differentiated and transparent criteria for the student 
workloads must be laid down m binding examination 
regulations. 

At least one introductory event should be held each 
semester when students are admitted. Each module for which 
credit points are awarded must be examined in the course of 
the semester (with the possibility of retaking the examination 
at the beginning of the following semester). 

The teaching staff must have a sufficient academic 
plurality. The profile of the course of study and the title of the 
degree must be clearly coherent. 



186 



C. CAMPBELL and C. ROZSNYAI 



Research and Application Orientation Complementing the 
Report of the AR (Reference Framework for Bachelor’s and 
Master’s Degree Courses): 



Research 

Oriented 



Application 

Oriented 



Bachelor’s Degree 

Teaching of 

Basic scientific principles (50%) 

Methods (20%) and 
knowledge of the subject (30%) 

As generalist research - oriented 
basic training for starting a 
professional career 

Ability to participate in research tasks 

Broad generalist basic knowledge 
Placement semester desired 
Subject oriented dissertation 
Ability to obtain further qualification 
In Master’s programme 
BF (research-oriented Bachelor’s 
Degree 
Teaching of 

Basic scientific principles (30%) 

Methods (20%) and 
Knowledge of the subject (50%) 

As generalist application-oriented 
basic training for starting a 
professional career 

Ability to take on responsible tasks in 
the field of development, application, 
sales 



Master’s Degree 

Teaching of 

Knowledge of the subject and 
methods (40%) 

Special knowledge (30%) and 
research abilities (30%) 

For more highly qualified professional 
tasks 

Comprehensive ability to participate 
actively in research tasks 
Theoretical and analytical abilities 

Ability to obtain further qualification 
in PhD programmes 
MF (research-oriented Master’s 
Degree) 

Teaching of 

Knowledge of the subject and 
methods (20%) 

Special knowledge (30%) and 
research abilities (30%) for more 
highly qualified professional tasks 



Comprehensive ability to participate 
actively in research tasks 



Theoretical and analytical abilities 



Ability to obtain further qualification 
in PhD programme 

MF (research oriented Master’s 
Degree) 

Note: Dependent on the previous degree courses. Master’s Degree courses can be designed 
as 

- Consecutive courses with regard to time and subject; 

Consecutive courses with regard to time, conversive courses with regard to subject, or 
complementary courses: 

Continuing education courses complementing a person’s special field (usually while 
working) and taking place after a break following the first degree. 



Broad generalist specialist knowledge 
Placement semester planned 
Application-oriented dissertation 

AbiUfy to obtain further qualification 
in Master’s programmes 

BA (application-oriented Bachelor’s 
Degree) 
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Bachelor’s Oe0-ees 


Master’s Degrees 


Admission requirements: 

As a rule, general university 


Admission requirements: 


entry qualification 
(BF - research-oriented 
Bachelor’s Degree) 


Bachelor’s Degree 


As a rule, restricted university 
entry qualification 

Relevant to All ^ - appBcatlon-orlented 

theProliles Bachelors Degree) 


In the case of continuing education 
courses: 

Two years of job experience 
Course length: as a rule, two to 


Course length as a rule, six to 


four semesters in the case of 


eight semesters 


continuing education courses: 
No restriction to course length 


Dissertation: 6 weeks to three 
months 


Dissertation: three to six months 


In the case of continuing education 
courses: no dissertation required 





Relevant to All the Qualifications 

Research orientation must be guaranteed by the scientific 
profile and by research activities of the academic staff. 

Application orientation must, in addition to the course 
contents and forms of study (placement semester, projects, 
application-orientated dissertations), be guaranteed by the 
scientific profile and by practice-oriented key areas as well as 
by development activities of the academic staff. 

Information provided by ZevA as of 12 March 2002. For 
further details contact ZevA <http://www.zeva.uni-hannover. 
de>. 

5.3. TOWARDS SUBJECT SPECIFIC DESCRIPTORS FOR 
QUALIFICATIONS: A NATIONAL APPROACH - THE QUALITY 
ASSURANCE AGENCY FOR HIGHER EDUCATION OF THE 
UNITED KINGDOM 

1 . Subject Benchmark Statements 

Subject benchmark statements provide a means for the 
academic community to describe the nature and characteristics 
of programmes in a specific subject. They also represent 
general expectations about the standards for the award of 
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qualifications at a given level and articulate the attributes and 
capabilities that those possessing such qualifications should 
be able to demonstrate. 

Subject benchmark statements are used for a variety of 
purposes. Primarily, they are an important external source of 
reference for higher education institutions when new 
programmes are being designed and developed in a subject 
area. They provide general guidance for articulating the 
learning outcomes associated "with the programme but are not 
a specification of a detailed curriculum in the subject. 
Benchmark statements provide for variety and flexibility in the 
design of programmes and encourage innovation "within an 
agreed overall framework. 

Subject benchmark statements also provide support to 
institutions in pursuit of internal quality assurance. They 
enable the learning outcomes specified for a particular 
programme to be reviewed and evaluated against agreed 
general expectations about standards. 

Finally, subject benchmark statements may be one of a 
number of external reference points that are drawn upon for 
the purposes of external re"vdew. Re"vdewers, however, do not 
use subject benchmark statements as a crude checklist for 
these purposes. Rather, they are used in conjunction "with the 
relevant programme specifications, that is, the internal 
evaluation documentation of the institution, in order to enable 
re"vdewers to come to a rounded judgment based on a broad 
range of e-vddence. 

The benchmarking of academic standards for this subject 
area has been undertaken by a group of subject specialists 
drawn from and acting on behalf of the subject community. 
The work of the group was facilitated by the Quality Assurance 
Agency for Higher Education, which publishes and distributes 
this statement and other statements developed by similar 
subject specific groups. 

In due course, but not before July 2005, the statement "will 
be re"vdsed to reflect developments in the subject and the 
experience of institutions and others who are working -with it. 
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The Agency will initiate revision and, in collaboration with, the 
subject community, will make arrangements for any necessary 
modifications to the statement. 

More than sixty subject benchmark statements have been 
produced in this manner. They are all published on the QAA 
web-site <http://www.qaa.ac.uk/cmtwork/benchmark/index.htm>. 

5.4. TUNING STRUCTURES IN HIGHER EDUCATION: 
LESSONS LEARNED IN HOW TO ACHIEVE RESULTS 

Tuning Educational Stmctures in Europe is a project funded by 
the European Commission and coordinated by the Universities 
of Deusto (Spain) and Groningen (The Netherlands). The pilot 
project has involved 105 university departments across Europe 
and has covered seven subject areas (five mainstream project 
participants and two “synergy groups”). The project differs from 
the Joint Quality Initiative in that it has a subject/discipline 
focus whereas the JQI is looking at the development of generic 
descriptors for BA/MA qualifications. 

The project is taking place in the context of the Bologna 
process which has stimulated debate and action (in varying 
degrees) within signatory countries to the Bologna Declaration, 
on issues such as the re-structuring of qualifications, 
programme design, and quality assurance. 

The aims of the Tuning project (see <http://www.relint. 
deusto.es/tuning> for further details) include: 

- opening up the debate with a range of stakeholders in 
higher education (graduates, students, and employers) 
on the nature and importance of subject-specific and 
general competencies; 

- identifying and exchanging information on common 
subject-based reference points, curricula content, 
learning outcomes, and methods of teaching, learning, 
and assessment: 

- improving European co-operation and collaboration in 
the development of the quality, effectiveness, and 
transparency of European higher education by 
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examining ECTS credits and other suitable devices to 
enhance progress. 

There is no intention that Tuning should lead to a unified, 
prescriptive, or definitive European curricula or that it should 
lead to the creation of a rigid set of subject specifications which 
might restrict or direct the content, delivery, or nature of 
higher education in Europe or to damage local and national 
academic autonomy. 

The final results of the project will be published by the end 
of 2002. The intention of this note is to focus on one of the 
objectives of the project - “to elaborate a methodology for 
analyzing common elements and areas of specificity and 
diversity in five subject areas: Business and Management, 
Educational Sciences, Geology, History, and Mathematics”. 
The methodology was designed on the basis of four lines of 
analysis of degree programmes with the aim of making them 
“more legible and transparent”. Such an approach would meet 
one of the objectives of the Bologna process that calls for the 
“adoption of a system of easily readable and comparable 
degrees”. The four lines are: 

Line 1: General and academic skills; 

Line 2: Knowledge, core curricula, and content; 

Line 3: ECTS as an accumulation system; 

Line 4: Methods of teaching and learning, assessment and 

performance, and quality. 

The purpose of this note is not to summarize the findings in 
respect of the objective but briefly to describe the steps that 
the participants took in order to work towards defining level 
descriptors for the first and second cycle studies in five subject 
areas in terms of knowledge, core curricula, and content (Line 
2). The project managers observed that there had been four 
phases of development: 

Phase 1 : Informing 

Phase 2: Storming 

Phase 3: Norming 

Phase 4: Performing 
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In Phase 1, within their subject groups, members informed 
each other about the current situation in their institutions 
concerning the tj^pes of programmes of study being designed 
and of future perspectives. Various S 5 mergy groups that had 
not been involved in the “informing” session then added to this 
information. 

Phase 2 was characterized by “questioning everything and 
anything”. Hot discussions and long hours were necessary to 
“get the steam out!” This attitude was also carried forward into 
the plenary discussions of the project Steering Group. 

In Phase 3, the subject groups identified what was common, 
diverse, and d 5 mamic in their subject areas. They tried to find 
a common framework for those elements for which it was 
useful to have clear reference points. At the same time, 
differences were highlighted and tested to determine whether 
they were in fact useful divergences and as such an 
enrichment. 

Finally, in Phase 4, the subject groups performed smoothly. 
Agreements were reached, ideas outlined, and everybody felt 
ready to take the project forward. There were strict deadlines 
to be met in the project, but all groups ended up able to 
present their results in a proper form and to make 
recommendations in regard to level descriptors in the five 
subject areas to a wider public at the final conference of the 
Tuning project on 31 May 2002. 

Success factors for this project to contribute to aspects of 
programme design appear to have included: 

- clear definitions of project aims and objectives; 

- a tightly defined time scale; 

- a small incentive for involvement of departments in 
terms of modest project funding; 

- freedom for academics to exchange views on subject- 
related competencies in a developmental environment; 

- cross-fertilization of ideas across the subject groups 
through plenary discussions and with stakeholders; 
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- dynamic subject group chairpersons who were not 
always from the subject group concerned thus not 
biased in terms of any particular approach to a subject; 

- effective project management. 

Similar projects have taken place at national level [e.g., in 
the United Kingdom - to develop subject benchmark 
statements). 



Part III 



References, Bibliographies, 
and Web Resources 



In addition to listing the references actually cited in the main 
text of this handbook, this section also includes several topical 
and thematic bibliographies as well as lists of Web resources. 
As an additional aid to the reader, some of the works listed are 
annotated. Given the overlapping of the topics and themes 
covered and the comprehensiveness of the references, a certain 
amount of repetition in the listing of resources was inevitable. 
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THE CENTRAL AND EASTERN EUROPEAN COUNTRIES i 

Billing, David, and Temple, Paul. Intermediary Bodies in 
Higher Education in Eastern Europe (Forthcoming). 

Council of Europe Legislative Reform Programme for Higher 
Education, 



^ For references to other books and studies on higher education issues in the Central and 
Eastern European Countries see <http://www.cepes.ro/publications>. 
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Advisory Mission, Riga, 24-26 October 1994 
<http://culture.coe.int> 

Council of Europe Legislative Reform Programme for Higher 
Education, Baltic States Co-operation for Quality 

Assurance in Higher Education. Report of the Advisory 
Mission on Quality Assessment and Accreditation, Riga, 19- 
22 February 1 995 <http: / /culture.coe.int> 

Council of Europe Legislative Reform Programme for Higher 
Education, Baltic States Co-operation for Quality 

Assurance in Higher Education. Statute on the International 
Co-operation in Quality of Assessment of Higher Education in 
the Baltic States, 1996 <http://culture.coe.int> 

Human Resources Development, Canada. Comparative 
Analysis of Higher Education Systems in Central and 
Eastern Europe. Ottawa: Canadian Bureau for International 
Education, 2000. 

Ten Years After and Looking Ahead: A Review of the 
Trcmsformation cf Higher Education in Central and Eastern 
Europe. Bucharest: UNESCO-CEPES, 2000. 

Tomusk, Voldemar. “Conflict and Interaction in Central and 
East European Higher Education: The Triangle of Red 
Giants, White Dwarfs, and Black Holes”, in. Tertiary 
Education and Management 3 3 (1997): 247-255. 

Tomusk, Voldemar. “WTien East Meets West: Decontextualizing 
the Quality of East European Higher Education”, Quality in 
Higher Education 6 3 (2000): 175-185. 

The World Bank. Hidden Challenges to Education Systems in 
Transition Economies. Washington D.C.: The World Bank, 
2000. 
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2.3. THEMATIC BIBLIOGRAPHIES2 

2.3. J. The Changing Context of Higher Education: Trends, 

Challenges, and Responses 

Adam, S. Transnational Education Report Brussels: CEURC, 
2001. [This report maps the development of transnational 
education (education which crosses national borders) within 
the European Union and makes several recommendations 
concerning the quality assurance of such provision.] 

Adelman, C. a Parallel Postsecondary Universe: The 

Certification System in Information Technology. Washington 
D.C.: United States Department of Education, Office of 
Educational Research and Improvement, 2000. 

Campbell, C., van der Wende, M. C. International Initiatives 
and Trends in Quality Assurance for European Higher 
Education. Helsinki: ENQA, 2000 <http://www.enqa.net>. 

Clark, B. R. Creating Entrepreneurial Universities: 
Organizational Pathways of Transformation. Oxford: 
Pergamon, 1998. 

Currie, J., and Subotzky, G. “Alternative Responses to 
Globalization from European and South African 
Universities”, in, N. Stromquist and K. Monkman. 
Globalization and Education: Integration and Contestation 
Across Cultures. New York: Rowman and Littlefield 

Publishers Inc., 2000. 

CVCP, and HEFCE. The Business of Borderless Education: UK 
Perspectives. London: Committee of Vice-Chaincellors and 
Principals of the Universities of the United Kingdom, and 
the Higher Education Funding Council for England, 2000. 
[A comprehensive account of the virtual aind corporate 
developments in learning which - along with the Internet - 
are changing the global landscape in the delivery of higher 
education services. The report alerts British universities 
and colleges to the challenge in borderless learriing posed 



2 These lists of publications are illustrative of the issues of quality assurance in higher 
education and related matters. In some cases, short summaries are included, but only for 
purposes of infonnation, and do not constitute any kind of recommendation. 
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by their overseas counterparts, particularly in North 
America. It addresses the question of how British 
universities may be able to rise to and capitalize on that 
challenge in the area of quality assurance. There is also a 
section on developments in Europe.] 

Dahrendorf, R. Universities after CornrnunisrrL Hamburg: 
Korber-Stiftung, 2001. 

Haug, G. and Tauch, C. Trends in Learning Structures in Higher 
Education. Helsinki: National Board of Education, 2001. 
[This report is a follow-up, prepared for the Salamanca and 
Prague Conferences of March and May 2001, respectively, 
on progress on the implementation of the Bologna process. 
See <http : / /www. oph . fi/pubUcations/ trends2> . ] 

Henicel, M. Academic Identities and Policy Change in Higher 
Education. London: Jessica Kinsley Publishers, 2000. 

Knight, J . Trade in Higher Education Services: The Implications 
of GATS. London: The Observatory on Borderless Higher 
Education, 2002 <http://www.obhe.ac.uk>. 

Larsen, K., Morris, R., and Martin, J. Trade in Educational 
Services: Trends and Emerging Issues - Working Paper. 
Paris: CERI/OECD, 2001. 

Maassen, P. Government Steering and the Academic Culture: 
The Intangibility of the Human Factor in Dutch and German 
Universities. Maarsen: DeTydstroom, 1996. 

Mallea, j. International Trade in Professional and Educational 
Services: Implications for the Professions and Higher 
Education. Paris: CERI/OECD, 1998. 

OECD. The Response of Higher Education Institutions to 
Regional Needs. Paris: OECD Publications, 1999. 

Scott, P., ed. The Globalization of Higher Education. 
Buckingham: Open University Press, 1998. 

Sporn, B. Adaptive University Structures: an Analysis of 
Adaptation to Socio-Economic Environments of US and 
European Universities. London: Jessica Kingsley, 1999. 

Task Force on Higher Education and Society. Higher 
Education in Developing Countries: Peril and Promise. 
Heemden, Virginia: World Bemk Publications, 2000. [The 
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Task Force on Higher Education and Society 
(<http://www.tfhe.org>) was convened by the World Bank 
and UNESCO to bring together experts from thirteen 
countries for the purpose of exploring the future of higher 
education in the developing world. Based on research and 
intensive discussion and hearings conducted over a two- 
year period, the Task Force has concluded that without 
more and better higher education, developing countries will 
have increasing difficulties in benefiting from the global 
knowledge-based economy. The report articulates the 
characteristics of an effective higher education system.] 

UNE^SCO. Higher Education in the Twenty-First Century: A New 
Commitment: Final Report of the World Conference on 
Science. Paris: UNESCO, 1998. 

The World Bank. Constructing Knowledge Societies: New 
Challenges for Tertiary Ekiucation. Washington D.C.: The 
World Bank, 2002. 

2.3.2 Governance and Management in Higher Education 

Barnett, R. Realizing the University in an Age of Supercomplexity . 
Buckingham: SRHE/Open University Press, 2000. 

Bauer, M., Marton, S., Askling, B., and Marton, F. 
Transforming Universities: Changing Patterns of Governance, 
Structure, and Learning in Swedish Higher Ekiucation. 
London: Jessica Kingsley Publishers, 1999. [This book 
examines the implications of reforms for a higher education 
system (Sweden) on three levels: the state, the institution, 
and the individual.] 

Braun, D., and Francois-Xavier, M., eds. Towards a New 
Model of Governance for Universities? A Comparative View. 
London: Jessica Kingsley, 1999. [This book draws on case 
studies in the United Kingdom, The Netherlands, France, 
Germany, Italy, The United States, and Switzerland, and 
demonstrates the vairied impact of increased state 
intervention in higher education. It provides a comparative 
overview of developments in the governance of universities 
since the 1980s and an assessment of whether and how 
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changes in state intervention have transformed the political 
and organizational management of universities.] 

Bleiklie, L, Hostaker, R, and Vabo, A. Policy and Practice in 
Higher Education: Reforming Norwegian Universities. 

London: Jessica POngsley Publishers, 2000. [This book 
describes the nature of change over the last thirty years of 
growth in the Norwegian higher education system.] 

Ford, P. Managing Change in Higher Education: A Learning 
Environment Architecture. Buckingham: Open University 
Press, 1996. [This study provides a generic model which 
can be used by managers of any individual learning 
institution in order to develop appropriate learning 
environment architecture of their own. It provides a method 
for creating a framework within which strategies, business 
processes, and supporting Information systems can be 
developed and changed to meet the objectives of an 
institution.] 

Kogan, M., and Hanney, S. Reforrning Higher Education. 
London: Jessica Kingsley, 1999. [The authors focus on the 
ways in which the changing concepts of the nature of the 
state and its role have had an impact on the development of 
higher education policy in the last thirty years. The authors 
study the d 3 mamics behind the shift from state-subsidized 
independence to ambiguous but increased dependence on 
and deference to state policies. The book looks at the 
changes in the machinery of the state for policy 
implementation and consequential changes in institutional 
governance.] 

OECD. Redefining Tertiary Education. Paris: Organization of 
Economic Co-operation and Development, 1998. 

2.3.3. Income Generation and Entrepreneurship 

Gray, H., ed. Universities and the Creation of Wealth. 
Buckingham: Open University Press, 1999. [This book 
provides a basic insight into what universities currently do 
in terms of the creation of wealth and into what kind of 
organizations they might usefully become. It explores how 
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universities benefit their local and regional communities, 
and what more they might do to thrive locally in the global 
economy.] 

2.3.4. The Evaluation of Institutional Performance 

Cave, M., Hanney, S., Henkel, M., and Kogan, M. The Use of 
Performance Indicators in Higher Ekiucation: The Challenge of 
the Quality Movement 3^^^ ed. London: Jessica Kingsley, 

1996. [This study provides a critical introduction to the 
basic concepts and practical considerations of using 
performance indicators within universities. Written 
primarily from a British perspective, with one chapter 
devoted to a comparative view, the third edition has been 
thoroughly revised to take account of the most recent 
developments in the practice, literature, and potential use 
of performance indicators with particular reference to 
developments in the “new” universities and colleges of 
higher education. Surveys of performance indicators 
relating to teaching, research, and adrninistration are 
presented, as well as a discussion on the various modes of 
application within various levels of an institution.] 

Smith, H., Armstrong, M., and Brown, S. Benchmarking and 
Threshold Standards in Higher Education. London: Kogan 
Page, 1999. [A wide ranging exploration of the issues and 
challenges involved in benchmarking and providing a 
critical analysis of the problems and potential of a powerful 
but controversial development tool in the United Kingdom, 
the United States, New Zealand, and South Africa.] 

Thackwray, B. Effective Evaluation of Training and 
Development in Higher Education. London: Kogan Page, 

1997. [Emphasis on the need for the development of better 
and more consistent practices with regard to evaluation. 
This book uses relevant examples and detailed case studies 
from the United Kingdom and elsewhere with the aim of 
providing a comprehensive guide to the best practice and a 
demonstration of the many benefits to be gained from the 
effective use of evaluation.] 
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2.3.5. Human Resource Management 

Farnham, D., ed. Managing Academic Stciff in Changing 
University Systems: International Trends and Comparisons. 
Buckingham: Open University Press, 1999. [This study 
provides a contemporary and international analysis of how 
academic staff in universities are currently managed. It 
reviews recent developments in higher education policy in 
fifteen selected countries and examines their impacts on the 
academic profession. While rates of change differ, the 
contributors argue that the massificatlon, marketization, 
and managerialization of higher education are universal, 
international phenomena.] 

2.3.6. Quality Assurance and Management^ 

Ashworth, A., and Harvey, R. Assessir^ Quality in Further and 
Higher Education. London: Jessica Kingsley, 1994. [This 
book outlines the system of Total Quality Management 
CTQM) in further and higher education, with illustrations of 
quantitative measurement for its evaluation. This handbook 
of techniques is intended for managers of universities and 
colleges.] 

Brown, S., Race, P., and Smith, B. 500 Tips for Quality 
Enhancement in Universities and Colleges. London: Kogan 
Page, 1997. [Aimed at staff in higher education institutions, 
this book sets out to provide helpful, practical, and realistic 
suggestions for the measurement and improvement of 
quality. It covers the full range of quality issues including: 
valuing students and staff; enhancing learning through 
teaching and assessment; quality processes, feedback and 
evaluation; preparing for assessment visits.] 

El-Khawas, E. “Quality Assurance in Higher Education: Recent 
Progress, Challenges Ahead”, Paper presented at the 



^ Copies of the publications listed in this section that are marked with an asterisk may be 
available from the Centre for Higher Education Research and Information (CHERI) or the 
Open University at <http://www.open.ac.uk/cheri>. 
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UNESCO World Conference on Higher Education, Paris, 
1998. 

European Commission. Evaluation of European Higher 
Education: A Status Report Prepared for the, DG XXII by the 
Center for Quality Assurance and Evaluation of Higher 
Education, Denmark, in co-operation with Comite National 
d’Evaluation, France. Paris, 1998 <http://www.enqa.net/ 
docs.lasso?docname=statusreport 1 .html>. 

Kristoffersen, Dorte, Sursock, Andree, and Westerheijden, 
Don. Quality Assurance in Higher Education: Manual of 
Quality Assurance: Procedures and Practices. Turin: The 
European Training Foundation, 1998. 

The European Training Foundation. Quality Assurance in 
Higher Education: A legislative Review and Needs Analysis 
of Developments in Central and Eastern Europe. Turin: The 
European Training Foundation, 1998. 

*The European Training Foundation. Quality Assurance in Higher 
Education: Final Report and Project Recommendations. Turin: 
The European Training Foundation, 1998. 

*The European Training Foundation. The European Dimension 
of Institutional Quality Management The European 
University. A Handbook on Institutional Approaches to 
Strategic Management, Quality Management, European 
Policy, and Academic Recognition. Turin: The European 
Training Foundation, 2000. 

Franke, S. “From Audit to Assessment: A National Perspective 
on an International Issue”, Quality in Higher Education 8 1 
(2002). [This article describes recent developments in the 
national quality assurance system in Sweden with the 
introduction of quality assessments of all higher education 
that leads to general and professional degrees.] 

Freed, J. E., and Klugman, M. Total Quality Management on 
Campus: Implementation, Experiences, and Observation. 
Phoenix: The Oryx Press, 1997. [This book analyses ‘Total 
Quality Management” (TQM) in action at ten colleges and 
universities across the United States, exploring the need to 
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improve quality, lessons learned, and the experiences of 
each institution implementing the various quality principles 
associated with the TQM philosophy.] 

Geddis, E. “A Perspective on Tensions between External 
Quality Assurance and Institutional Quality Development: A 
Case Study”, Quality in Higher Education 8 1 (2002). 

HAmAlAinen, K., et al. Quality Assurance in the Nordic Higher 
Education - Accreditation-like Practices. Helsinki: ENQA, 
2001. 

Harvey, L., and Green, D. Dejining Quality: Assessment and 
Evaluation in Higher Education 18 1 (1993). 

Kells, H. R. Self-Regulation in Higher Education: a 

Multinational Perspective on Collaborative Systems of Quality 
Assurance and Control London: Jessica Kingsley, 

1992. [Intended as a reference source and guide for policy 
makers, this book describes several major forms of 
regulatory systems for higher education. Existing models 
and systems are examined in national contexts. Useful 
appendices include excerpts from various national and 
regional systems.) 

Jones, D. P. Different Perspectives on Information about 
Educational Quality: Implications for the Role of 

Accreditation. Washington D.C.: Council for Higher 

Education Accreditation, 2002. 

Liston, C. Managing Quality and Standards. Buckingham: 
Open University Press, 1999. [Quality management, the 
application of standards, and client service areas are as 
vitally important to higher education as they are in any 
other enterprise or public service activity. This text provides 
a practical, common sense approach to using procedures 
and information to demonstrate improved performance and 
to account for outcomes. The text confronts both the quality 
jargon and the C 3 micism of many academics about the 
“quality police”. It is full of practical examples and 
guidelines for action and is useful for managers in 
universities and colleges worldwide.) 



ERIC 




RESOURCES 



215 



Lxdmas, L. “Does the Development of Mass Education 
Necessarily Mean the End of Quality?”, Quality in Higher 
EkiucationS 1 (2002). 

Middlehurst, R. Quality Assurance Implications of New Forms of 
Higher Ekkwalion Helsinki: ENQA, 2001 <http://www.enqa.net>. 

Seddon, J. The Case Against ISO 9000. 2^^ edition. Dublin: 
Oak Tree Press, 2000. 

Smeby, J. C., and Stensaker, B. “National Quality Assessment 
Systems in the Nordic Countries: Developing a Balance 
Between External and Internal Needs”, Higher Education 
Policy 12 (1999): 3-14. 

Smith, D. “How Will I Know If There Is Quality?” Report on 
Quality Indicators and Quality Enhancement in Universities. 
Toronto: Council of Ontario Universities, 2000. [This report 
proposes that quality indicators should be considered as 
only a component of a broader, more powerful approach to 
quality enhancement. They reveal certain symptoms of 
quality, but they do not show the causes of quality nor the 
conjunction of resources and incentives that produce 
quality. This report concludes with an emphasis on 
fostering a competitive and collaborative environment for 
universities, within a university sector that is funded 
reasonably, in line with competing university sectors.) 

Strathern, M. Audit Cultures: Anthropological Studies in 
Accountability, Ethics, and the Academy. London: Routledge, 
2000. [In this book, twelve social anthropologists from 
across Europe and the Commonwealth chart the 
controversial phenomenon of “accountability”.] 

Tomusk, V. “When East Meets West: Decontextualizing the 
Quality of East European Higher Education”, Quality in 
Higher Education Q 3 (2000): 175-185. 

Van Damme, D. ‘Trends and Models in International Quality 
Assurance and Accreditation in Higher Education in 
Relation to Trade in Education Services”, Paper presented 
at the OECD/US Forum on Trade in Educational Services, 
23-24 May 2002. 



216 



C. CAMPBELL and C. ROZSNYAI 



Vroeijenstein, a. I. Improvement and Accountability: 
Navigating between ScyUa and Charybdis. London: Jessica 
Kingsley, 1994. [This study provides a guide for the design 
of systems of external quality assessment, drawing upon a 
model developed in the Netherlands but recognizing that no 
one model will meet the needs of different systems of higher 
education.] 

Westerheijden, D. F. “'Ex oriente lux?’ National and Multiple 
Accreditation in Europe after the Fall of the Wall and after 
Bologna”, Qucdity in Higher Education 7 1 (2001): 65-75. 

2.3.7. Open and Distance Learning 

Harry, K. Higher Education Through Open and Distance 
Learning, World Review of Distance Education and Open 
Learning Series. London: Farmer Press, 1999. [Open and 
distance learning has grown substantially in recent years 
across all levels of education, disciplines and countries. 
This book looks at the state of the art in open and distance 
learning and presents an educational culture in transition. 
The edited collection contains authoritative anadyses of key 
issues together with current accounts of practice in each 
region of the world.] 

Hope, a. “Quality Assurance”, in, G. M. Farrell, ed. The 
Changing Face of Virtual Educatiorv The Commonwealth of 
Learning. Vancouver, British Columbia, 2001 <http: 
/ /www.col. org/virtualed/ > . 

IHEP. Quality on the Line: Benchmarks for Success in Internet- 
Based Distance Education. Washington D.C.: Institute for 
Higher Education Policy, 2000 <http://www.ihep.com/ 
quality.pdf>. 

Loane, S. Distance Education and Accreditation. Washington 
D.C.: ERIC, 2001 <http://wwweriche.org/digests/2001- 
08.htm>. 

Oecd. E-leaming: The Partnership Challenge. Paris: 

Organization for Economic Co-operation and Development, 

2001. 
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3. SOME USEFUL WEBSITES 

- Central and Eastern Europe Network: 
<http://www.ceenetwork.hu> [individual agency links 
(access from <http://www.ceenetwork.hu>) [information on 
CEE agencies also at <http://www.inqaahe.nl>] 

- European Network for Quality Assurance (ENQA): 
<http://www.enqa.net> [This site is a comprehensive one 
having links to all member organizations. All ENQA 
publications, including the newsletter, are available on the 
site and can be downloaded free of charge. Also available 
are details of forthcoming workshops and seminars that are 
open to non-members as well as to members.) 

- International Network of Quality Assurance Agencies of 
Higher Education (INQAAHE): <http://www.inqaahe.nl> 
[This site has links to all member organizations. The 
network newsletter can be downloaded from the site free of 
charge.) 

- Council for Higher Education Accreditation - USA (CHE^): 
<http://www.chea.org> [This site has links to all CHEA - 
recognized accreditation bodies in the United States as well 
as a useful international database and glossary.) 

- Akkreditierungsrat (Germar^: 
<http://www.akkreditierungsrat.de> [This agency is 
approximately the German equivalent of CHEA. It only 
accredits programmes itself on special request otherwise its 
main role is to accredit regional or subject specific 
accreditation agencies. It has links to other German 
accreditation agencies.] 

3.1. INTERGOVERNMENTAL ORGANIZATIONS 

- Council of Europe: <http://www.coe.int> 

- European Commission: 

<http: / / www.europa.eu.int/ comm. / education/higher. html> 

- OECD: <http://www.oecd.org> 

- UNE^SCO: <http://www.unesco.org> and European Centre 
for Higher Education: <http://www.cepes.ro> 
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- World Conference on Higher Education: 
<http://www.unesco.org/education/educprog/wche.index. 
html> 

- Global Forum for Quality Assurance, Accreditation, and the 
Recognition of Qualifications: 
<http://www.unesco.org/education/studyingabroad> 

- World Bank: <http://www.worldbank.org> 

- World Trade Organisation: <http://www.wto.org> 

3.2. NON-GOVERNMENTAL ORGANIZATIONS 

- American Association for Higher Education (AAHE): 

<http:/ /WWW. aahe.org> 

- American Council for Education (ACE): 
<http://www.ace.org> 

- Association of Commonwealth Universities (ACU): 
<http://www.acu.ac.uk> 

- Center for Higher Education Research and Infofmation 
CHERI: <http:/ /www.open.ac.uk/cheri> [See information 
and new project “Universities and Their Role in the 
Transformation of Society”.] 

- European Universities Association (EUA): 
<http://www.unige.ch/eua> 

- International Association of Universities (lAU) : 
<http://www.unesco.org/iau> 

- National Union of Students in Europe (ESIB) : 
<http://www.esib.org/> 

- Universitas2 1 : <http://www.universitas21.com> 

- Coimbra group: <http:// www.coimbra-group.pt> 

3.3. SUBJECT ACCREDITATION/RECOGNITION AGENCIE:S 

OPERATING ON AN INTERNATIONAL BASIS 

3.3. 1 . Business and Management 

- Association of MBAs (AMBA): <http://www.mba.org.uk> 
[Accredits MBA programmes in the UK and abroad] 
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- American Assembly of CoUegiate Schools pf Business 
(AACSBIntemational): <http://www.aacsb.edu> [Accredits 
Business, Management, and Accounting Schools in the USA 
and in eleven other countries] 

- European Foundation for Management Development 
(EFMD): <http://www.efmd.be> [Manages the EQUIS 
Programme and accredits business schools and beyond.] 

- Foundation for International Business Administration 
Accreditation (FIBAA): <http://www.mba-info,de> [This 
organization is accredited by the Akkreditierunsgrot 
(Germany) and accredits programmes in Business and 
Management in Austria, Germany, and Switzerland.] 

3.3.2. Engineering 

- European Federation of National Engineering Associations 
(FEANI): <http://www.feani.org> 

- The Washington Accord: <http: / /www. washingtonaccord.org/ > 

3.4. SOME OTHER RESOURCES ON ASPECTS OF QUALITY 

AND/OR EDUCATION 

- Bized <http://catalogue.bized.ac.uk> 

- European Foundation for Quality Management (EFQM) 
<http: / /www.efqm.org/new_website> 

- Information on Malcolm Baldridge, TQM and EFQM: 

<http: / /www. qpronline.com/baldridge/index.html> 

- International Organization for Standardization (ISO) 
<http://www.iso.ch/en> 

- National Institute of Standards and Technology (USA) 
<http://www.quality.nist.gov/> 

- Observatory on Borderless Higher Ekiucation: <http:/ / 
www.obhe.ac.uk>. [An international strategic information 
service tracking developments in on-line learning, 
transnational provision, and new providers {e.g., for-profit 
universities). The service is designed to appeal to a wide range 
of organizations and individuals concerned with higher 
education policy.] 
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UNESCO-CEPES PUBLICATIONS 

The UNESCO European Centre for Higher Education produces five 

series of publications: , , i_ j . 

- the quarterly review, ffigher Education in Europe, pubUshed m 

three language versions: English, French, and Russian; 

- the Studies on Higher Education, which present relatively 
comprehensive reports on and analyses of major issues in higher 
education; 

- the Papers on Higher Education, which present shorter studies 
and occasional papers; 

- the Monographs on Higher Education, which present studies on 
national systems of higher education according to a common outline, 

- the Studies on Science and Culture, that publish the research 
findings undertaken foremost by the UNESCO Chairs, which are 
collaborating with UNESCO-CEPES, in subject areas other than higher 
education. 



HOW TO ORDER 

Subscriptions to the English version of Higher Education in Europe 
must be placed directly with CARFAX: Carfax Publishing, Taylor & 
Francis Ltd.; Customer Services Department; Ranking Road; 
Basingstoke, Hants RG24 8PR, United Kingdom; E-mail: 
enquiiy@tandf.co.uk. The French and Russian versions can be obtained 
free of charge through the UNESCO-CEPES Web page: 
http://www.cepes.ro 

For the volumes in the other series, please check off the titles, that you 
would like to purchase bearing in mind that volumes in the Studies..., 
the Monographs..., and the Studies on Science and Culture series cost 
$20.00 (USD) or €22,00 each. Each volume in the Papers... series costs 
$15.00 [USD] or €16.50. 

Please fill in the following blanks: 

Your Name: 

Institution: 

Address: 



Telephone: leieiax. 

E-Mail: 

Please mail this information to: 

The Senior Editor 

UNESCO-CEPES 
39 Strada $tirbei-Voda 
RO-70732 Bucharest, Romania 

We shall bill you for the material that you have ordered and include 
Instructions as to payment. 



Date: 



215 



er|c 



Studies on Higher Eklucation 

* Gains and Losses: Women and 
Transition in Eastern and Central 
Europe (English. 1994, 115 p.) 

* Academic Freedom and University 
Autonomy. Contributions to tlie 
International Conference, 5-7 May 1992, 
Sinaia (English, 1993, 309 p.) 

* Higher Education Reform in Romania: A 
Study (English, 1992, 143 p.) 

* The Doctorate in the Europe Region 
(English, 1994, 225 p.) 

* Standards and Diversity in Architectural 
Education (English, 1996, 353 p.) 

* Ten Years After and Looking Ahead: A 
Review of the Transformations of Higher 
Education in Central and Eastern 
Europe (English, 2000, 410 p.) 

* Transnational Education and the New 
Economy: Delivery and Quality (English, 
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CEPES 



UNESCO-CEPES (European Centre for Higher 
Education /Centre Europeen pour I'Enseignement 
Superieur), a decentralized office of the UNESCO 
Secretariat, was established in September 1972 with a 
view to promoting co-operation in higher education 
among Member States of the Europe Region (the 
countries of Europe, North America, and Israel). In 
addition to this major mission, UNESCO-CEPES also 
serves as a field office representing UNESCO in 
Romania. 



Although the activities of UNESCO-CEPES are focused foremost on 
higher education in the Europe Region, the Centre also maintains 
contacts with relevant organizations and institutions in other regions, 
in conformity with the universal vocation of UNESCO. 

Through its pan-European mission and specific competence and 
experience in Central, South-East, and Eastern Europe, developed 
over thirty years of presence in the region, UNESCO-CEPES is, in its 
own way, a unique institution that deals with higher education in the 
Europe Region by providing bridges for active international co- 
^ operation. 

UNESCO-CEPES is focusing its activities along the following four 
^ main strands: 

- Policy and Reform of Higher Education; 

- Inter-university Co-operation and Academic Mobility; 

- Publications, Studies, and Information Services; 

- Status of Teachers and Teaching/ Learning in the Information 
Society. 

It also: 

- provides consulting services; 

- participates in the activities of other governmental and non- 
governmental organizations; 

- serves as a link between UNESCO Headquarters and Romania in 
relation to the activities of the Organization in Romania. 

In order to respond to the need for topical actions in support of 
the processes of reform and development of higher education in 
Central and Eastern Europe, including South-East Europe, UNESCO- 
CEPES has focused its current activities on the making and 
implementation of higher education policy, legislative reforms in 
education, academic quality assurance and accreditation, and the 
recognition of academic and professional qualifications. 

It is also concerned with new approaches to governance and 
institutional management, university autonomy and academic 
freedom, the status of teachers and their training, university-industry 
relations, the use of new information technologies, including the 
Internet/virtual university, and transnational education. 
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